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OUTLIER  RESISTANT  FILTERING  AND  SMOOTHING 

by 

Haralampos  Tsaknakis  and  P.  Papantonl-Kazakos 
University  of  Connecticut 
EECS  Department,  U-157 
Storrs,  Connecticut  06268 

Abstract 

We  consider  a  stationary  Gaussian  Information  process  transmitted  through  an 
additive  noise  channel.  We  assume  that  the  noise  and  information  processes  are 
mutually  Independent,  and  we  model  the  noise  process  as  nominally  Gaussian  with 
additive  Independent  oullers.  For  the  above  system  model,  we  first  develop  a  theory 
for  outlier  resistant  filtering  and  smoothing  operations.  We  then  design  specific 
such  nonlinear  operations,  and  we  study  their  performance.  The  performance  criteria 
are  the  asymptotic  mean  squared  error  at  the  Gaussian  nominal  model,  the  breakdown 
point,  and  the  Influence  function.  We  find  that  our  operations  combine  excellent 
at  the  nominal  model  performance  with  strong  resistance  to  outliers. 
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1.  Introduction 


In  filtering  and  smoothing,  information  carrying  data  are  extracted  from  noisy 
observations.  The  formalization  and  solution  of  the  filtering  and  smoothing  problems 
are  well  established,  when  the  joint  process  that  characterizes  the  relationship  be¬ 
tween  information  and  noise  data  sequences  is  statistically  well  known  (see  Kalman 
(1960,  1963),  Kolmogorov  (1941),  and  Wiener  (1949)),  or  parametrically  known. 

Linear  filtering  and  smoothing  operations  are  then  by  far  the  most 
widely  used,  due  to  their  simplicity  in  implementation.  In  practice,  however, 
the  occurence  of  occasional  extremely  erroneous  data  values,  called  outliers,  are  fre¬ 
quently  observed.  Furthermore,  linear  data  operations  are  notoriously  nonreslstant 
to  such  outliers,  inducing  dramatic  performance  instabilities.  The  purpose  of  this 
paper  is  to  establish  a  theory  for  outlier  resistant  filtering  and  smoothing  procedures 
and  to  provide  specific  such  data  operations  for  Gaussian  information  processes,  and 
additive,  nominally  Gaussian,  noise  processes.  Our  presentation  is  based  on  the 
theory  of  qualitative  robustness  (see  Boente  et  al  (1982),  Cox  (1978),  Hampel  (1971), 
Papantonl-Kazakos  and  Gray  (1979),  and  Papantoni-Kazakos  (1981,  1983,  1984a,  1984b)). 

2.  Preliminaries 


Let  us  consider  discrete-time  information  and  noise  stochastic  processes.  Let 


=  {X  , . .  ,X,  }  ,  =  {W. , . .  ,W,  }  ,  and  =  {Y.  , . .  .  ,Y,  }  ,  i>k,  denote  random  data 

1  i  k  i  i  k  i  1  k  — 

sequences  respectively  generated  by  the  information,  noise,  and  observation  (joint 


information  and  noise)  stochastic  processes.  Let  x*5  =  {x  , . .  ,x.  }  ,  w.  =  {w.,..,w  }, 
and  y^  =  {yj^,..,yy^}  ,  i>k,  denote  realizations  of  respectively  the  random  sequences, 
X^,  W^,  and  Y^,  and  let  x^,  w^,  and  y^  all  take  their  values  on  the  Euclidean  space 
Let  y  denote  the  measure  of  the  observation  process,  and  let  denote  the 


n-dimensional  restriction  of  y.  Let  the  objective  be  to  estimate  the  information 
datum  x^,  from  the  observation  sequence  y^,  where  i^^^k,  and  let  the  moan  squared 
criterion  be  used.  If  the  measure,  y,  is  well  known,  then  the  mean  squared  estimate. 


x^,  of  is  given  by  the  conditional  expectation  E  If  the 

measure  y  is  also  Gaussian,  then  the  latter  conditional  expectation  is  a  linear 
transformation  of  the  observation  sequence  y^.  Given  the  measure,  y,  and  the 
observation  sequence,  y^,  the  mean  squared  estimate,  x^  =  x^Cy^)*  Is  a  function 
of  the  sequence  y^,  whose  specific  form  is  determined  by  the  measure  y.  The  Induced 
mean  squared  error,  ^s  then  equal  to  the  expected  value 

E{[X^-x^(Y^)  ]2  |yk-i+l},  where  x^(y^)  =  E{X^|y^,y*^"^'’'^} . 

The  occurence  of  occasional  outliers  Induces  uncertainties  in  the  description 
of  the  measure  y.  The  initial  issue  is  then  the  qualitative  characterization  of 
those  uncertainties,  with  the  final  objective  being  the  design  of  outlier  resistant 
filtering  and  smoothing  operations.  As  It  has  been  previously  established  (Boente 
et  al  (1982)  and  Papantonl-Kazakos  (1983,  1984b)),  the  outlier  model  is  best  described 
by  a  Prohorov  class.  In  particular,  if  y^  denotes  the  nominal  measure  of  the  observa¬ 
tion  sequences,  the  outlier  model  is  described  by  the  Prohorov  ball  II  (y  ,y)£e  of 

>  Pjj  o 

processes,  where  e  represents  frequency  of  outlier  occurence,  and  where  p^(x”,y”)  is 
a  distortion  measure  between  data  sequences  x^  and  y”  respectively  generated  by  the 
measures  y^  and  y,  defined  as  follows: 

^  ;  for  given  finite  n 

(1) 

inf  {a:n“l(//i:Y^(x^^,y^^)>a]<a};  for 

n>n  ,  where  m  and  n  are  fixed  positive  integers 
o  o 

If  N  denotes  the  class  of  joint  processes,  v,  whose  marginals  are  y^  and  y,  then 

the  Prohorov  distance  11  (y  ,y)  is  defined  as  follows; 

n  9  pfi  o 

n  (y  ,y)  =  inf  inf  {5 :v()fjy’^:p  (x?yO)  >6)<6}  (2) 

n,Pn  0  n  - 

veW 

The  distortion  measure  p^(xOy'^)  in  (1)  is  clearly  a  metric,  for  all  n,  the 
inte>;er  m  is  a  design  parameter  corresponding  to  outlier  patterns,  and  n^  is  an 
integer  determined  by  the  nominal  measure  i^.for  the  satisfaction  of  performance 
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i+fi— 1 

stability.  In  particular,  given  n,  an  estimate,  x^  (y^  ),  1^,  of  the 

information  datum  is  called  outlier  resistant  or  qualitatively  robust  at 

Uo.  iff: 

Given  e>0,  there  exists  6>0,  such  that. 

Furthermore,  for  the  estimate  x^  (y^"*^”^),  1^  to  be  outlier  resistant  at  iJ^,for 

/S 

given  finite  n,  it  is  sufficient  that  x^  be  bounded,  and  that  the  following  continuity 
condition  holds  (see  Papantoni-Xazakos  and  Gray  (1979) ,Boente  et  al  (1932), and  Papanto 
Kazakos  (1981,  1983,  1984a,  1984b)),  if  the  class  H  in  (2)  includes  stationary 


processes  only. 


Condition  A 


For  given  finite  n,  polntwlse  continuity  as  a  function  of  the  data. 

That  is,  given  e>0,  given  xl-"*^"^,  there  exists  6>0,  such  that, 
i+n-1 

yi+n-l.  n“ir  IXj'Yj  l'^*5  implies  lx£(xf+'’~^)-jJ^(yf'^“^)  |<e 
If  W  is  a  class  of  stationary  and 

ergodlc  processes,  then  the  limit  11m  H  _  (u  ,p)  of  the  Prohorov  distance  in  (2) 

n-K»  ”*Pn  o 

equals  the  Prohorov  distance  JT  (p  ,u),  where  Y  (x"',y®)  is  as  in  (1).  In  add:,.- 

®*Tin  ®  TO 

tion,  if  the  measure  p®  is  absolutely  continuous,  and  if  then  f®(x®)  denotes  the 

°  ,  ,  o 

density  function  induced  by  the  measure  p®  at  the  vector  point  x®,  the  class  P” 

of  density  functions  defined  below  is  contained  in  the  class  IT  (p  ,p)<e  of 

®  >Yin  ® 

measures  p. 


P”  =  {  f®  =  (1-e)  f®  +  eh®  ,  h®  any  m-dimensional  density  function}  (3) 
The  class  P”  of  densities  in  (3)  represents  the  occurence  of  arbitrary  outlier 
m-dimensional  data  sequences,  with  probability  e,  where  with  probability  1-e  each 
m-dimensional  data  sequence  is  generated  by  the  nominal  measure  p^. 


-  4  - 

3.  The  Model  and  Outline  of  the  Approach 

We  vrill  adopt  the  nominal  model  of  additive  and  mutually  independent  informa¬ 
tion  and  noise  processes,  which  possess  density  functions,  for  any  given  dimenslonalit 
n.  Given  n,  these  density  functions  are  respectively  denoted  f"^  and  f^,  and  the 
corresponding  density  function,  f",  of  the  nominal  observation  process  is  then  given 
by  the  convolution  f”^  *^oN‘  will  consider  m-dimenslonal  outlier  patterns,  as 
described  by  the  class  F"*  in  (3),  with  f™  =f®  *  f™„.  Given  the  above  model, 

we  wish  to  estimate  the  data  sequences, ..  ,x  ,  generated  by  the  density 

function  In  particular,  each  information  datum  x^  is  estimated  via  an  observa¬ 

tion  sequence  yl'^*^“^,l<;£._<l+k-l,  where  if  £“i+k-l  the  estimation  operation, 

corresponds  to  filtering,  and  where  it  corresponds  to  smoothing  otherwise. 

Initially  adopting  the  mean  squared  criterion,  we  are  seeking  filtering  and 
smoothing  operations  that  are  outlier  resistant,  and  simultaneously  Induce  satisfactory 
performance  at  the  nominal  model. For  finite  number  of  obsetvafions,  condition  A, section 
in  conjuction  with  boundness  are  sufficient  for  outlier  resistance.  They  define, 
however,  a  large  class  of  possible  operations,  and  they  do  not  incorporate  perform¬ 
ance  at  the  nominal  model.  We  will  address  the  overall  issue  (outlier  resistance 
and  performance  at  the  nominal  model)  via  a  combination  of  saddle  point  game  theory 
and  the  theory  of  qualitative  robustness.  We  will  do  this  in  two  steps:  (a) 
when  each  information  datum  is  estimated  via  a  length  -k  observation  sequence,  for 
k^m,  where  m  as  in  (3),  and  (b)  when  the  latter  length  k  exceeds  ra.  At  each  step, 
a  saddle  point  game  is  first  formalized  and  solved.  Then,  the  induced  by  the 
latter  solution  filtering  or  smoothing  operation  is  checked  against  the  continuity 
condition  A  in  section  2  and  against  boundness.  If  those  conditions  are  not 


satisfied,  the  operations  are  appropriately  modified.  To  avoid  vagueness  in  our 
presentation,  we  assume  from  now  on  that  the  nominal  information  process  (with  density 
function  f  )  is  zero  mean  and  Gaussian. 


-  5  - 

4 .  Construction  of  Filtering  and  Smoothing  Operatlons-Step  1 

Consider  the  model  In  section  3,  with  f  Gaussian,  zero  mean,  and  not  necessaril 

os 

stationary.  Let  then  R^(l,j)  denote  the  autocovariance  matrix  of  the  random  informa¬ 
tion  data  sequence  x|,  where  j^i,  and  consider  the  mean  squared  estimation  of  the 

<+k— 1 

the  information  datum  x^,  using  an  observation  sequence  y“  ,  such  that  l£££l-t-k-l 
and  k<m,  where  m  is  as  in  (3) .  For  dimensionalities  k^m,  the  observation  sequence 

is  generated  by  the  k-dlmensional  restriction  of  some  density  function  f® 
in  the  class  F®.  Given  some  estimate,  ,  of  the  datum  x^,  the  mean  squared 

error,  e^  ^^j^_^(f“‘,X£)  ,that  it  induces  at  the  density  f®, equals  e{  ]^  |  f®} 

Given  k£m,  we  then  consider  a  saddle  point  game,  with  payoff  function  the  error 
e^  particular,  we  search  for  a  pair  (f“,X£)  such  that  f^  eP“,  and, 

Vf®eF®;  l®i,i+k-l 

From  the  results  in  Papantonl-Kazakos  (1984a) ,  we  conclude  that  for  the  class 

in  (3),  the  game  in  (4)  has  a  saddle  point  (f™,X£).  From  the  theory  of  saddle  point 
games  we  then  conclude  that  the  saddle  point  can  be  found  as  follows. 

(f®,x*)  :  (f“,xj)  inf  e.  (f”,x^)  (5) 

;where,  if  f*^  denotes  the  k-dimensional  restriction  of  the  density  f®,  then, 

a|  =  E{X|}  (7) 


Kf*^)  =  E{  e2{X^|  }  I  f*^; 


; and  thus , 

f®  :  I  (f^)  =  inf  Kf*^)  (9) 

fmeF® 

xj  :  =  E{  X^l  yJ■^‘'"^f^J)  (10) 

Let  us  now  denote  by  the  (£-1+1) th  row  of  the  autocovariance  matrix 
R^(i,i+k-l);  that  is,  =  [E{X^X^}  ,. . .  ,E{Xj^^j^_j^X^}  1 .  Let  now  (yl‘^'<^"^) 

denote  the  directional  derivative  of  f*^(yl'*‘^~l)  with  respect  to  the  column 
vector  V^.  Then,  since  the  information  process  is  zero  mean  Taussian,  we  easily 
find  that  the  quantity  Kf*^)  in  (8)  takes  the  following,  form,  wlu’re  K  denoti's 


the  real  line. 
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,  f  [V,fk(yi+k-l)]2 
)  =  /  -  -  ^  ^ -  d  yi+k-1 


The  functional  in  (11)  is  a  generalized  Fisher  information  measure,  and  it 

is  vaguely  lower  semicontinuous  in  the  vague  topology  of  all  the  k-dlmenslonal  density 

functions  (Huber(3981)) .  In  addition,  the  closure,  F™,  of  the  class  P®  in  (3) 

c 

is  vaguely  compact.  Therefore,  the  infinum,  inf  I(f^),  exists  and  is  attained 

£mpm 

in  If  f™  is  some  density  in  that  attains  the  latter  infimum,  then  the 

estimate  x^  in  (10)  takes  the  following  form. 

7„fk(yi+k-l) 

f*(yi  ) 

Regarding  uniqueness,  we  proceed  with  the  following  theorem,  whose  proof 
is  in  appendix  A. 

Theorem  1 

Let  f^  and  f^  be  two  densities  in  F®  that  both  attain  the  infimum 

inf  Kf*^).  Then, 
f“eF“ 


V^ff(yi+‘^-i)  V^fk(yi+k-l) 

,k.  i+k-1.  ^k,  i+k-1. 

£l<yi  )  fjCyj  ) 


;  a.e.  gr'^ 

^  1 


saddle  point  estimate  x^  in  (12)  is  thus  a.e.  unique  in  P°. 


To  find  the  explicit  form  of  a  saddle  point  pair  (f*,X£),  we  will  now  assume 
that  the  nominal  noise  process  is  zero  mean  Gaussian  and  we  will  denote  the  auto¬ 
covariance  matrix  of  the  random  sequence  W3,j>i  from  this  process,  Rj^(i, j)  .  Con¬ 
sidering  the  (£-i+l)  th  row  V-  of  the  autocovariance  matrix  R  (i,i+k-l),  defined 
earlier,  we  then  denote. 


7 


”ik  "  \  (l.i+k-1)  +  (l,i+k-l) 


‘'ik«>  ■  »Ik  ''I 


^k«>  ■  h  ■’ik  «> 


k  i+k-1 

y  =  yi 


Denoting  by  (j)(x)  and  4>(x)  respectively  the  density  function  and  the 
cumulative  distribution  at  the  point  x,  of  the  zero  mean,  unit  variance  Gaussian 
random  variable,  we  then  express  the  following  lemma,  whose  proof  is  in  appendix  A. 


Lemma  1 

Let  the  nominal  information  and  noise  processes  in  class  F®  be  both  Gaussian, 
with  zero  mean  and  autocovariance  matrices  as  above.  Then,  a  saddle  point  solution 
(f^jX^)  of  the  game  in  (4)  is  given  by  the  expressions  below,  where  x^(y^)  is  unique 
a.e.  in  R  ,  and  where  ]  1 ,  T,  (-1)  respectively  denote  determinant,  transpose,  and 


inverse. 


k  k 

f*  (y  ) 


(1-e)  exp|-  I 


(1-c)  (2Tr)“^^^  exp 


|-  I  y*"  M-J  y‘'|;  y*^  :  £  A 

(a^  ifkVi  k 


I  k  ,  T  k, 

- 1  =  y  =  I  "  ^ 


^  k 

(y  ) 


_T  k  k  i„T  k]  , 

^Ik  y  y  :  IP^I^  (^)  y  I  1  A 


A  sgn  P.^  (O 


k  I  T  „  k  I 
y  :  |P.j^  (n  V  j 
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;  where, 


X  =  c 


(16) 


c 


<t>  (c)  +  c 


4)  (C) 


2-c 

2(l-c) 


(17) 


We  note  that  given  e,  the  constant  c  in  (17)  is  unique.  Also,  r,,(£)=I(f  ), 

X  K  0 

where  is  the  nominal  Gaussian  observation  density.  Thus,  given  t  and  i, 
is  monotonically  nondecreasing  with  increasing  k.  Similarly,  given  t  and  k, 
rik(^)  is  monotonically  nondecreasing  with  decreasing  i.  We  observe  that  the 
estimate  in  (15)  is  a  truncated  version  of  the  linear,  optimal  at  the  nominal 
Gaussian  model,  mean  squared  estimate.  In  addition,  for  E-»0,  the  constant  c  in  (17), 
tends  to  infinity,  and  so  does  then  the  truncation  constant  X.  In  the  latter  case, 
the  estimate  in  (15)  becomes  identical  to  the  optimal  at  the  nominal  Gaussian 
model  mean  squared  estimate.  Finally,  for  any  c>0  in  (17),  the  estimate  in  (15) 
is  clearly  bounded,  and  satisfies  condition  A  in  section  2;  it  is  thus  outlier 
resistent. 

ni  ^  ^ 

The  mean  squared  error,  e^  j^(f^,x^),  induced  by  the  estimate  in  (15), 

at  the  least  favorable  in  density  function  fj^(y'^)  in  (14)  , equals  0£-I(f^)  , 
where  I(f  )  is  the  information  measure  in  (11).  The  above  error  Is  the  largest  mean 
squared  error  induced  by  x^  in  F*",  and  by  substitution  we  obtain. 


®i,i+k-*l 


}  l-(l-r)  (2  <1  (c)  -l) 


(18) 


;  wiiere  r  is  as  in  (17),  .md  r  (f)  is  as  in  (11),  atul  whorf. 
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Let  the  mean  squared  error  induced  by  the  estimate 

Xp  in  (15)  ,  at  the  nominal  Gaussian  observation  density  f"*,  and  let  e°  (£) 

^  o'  i,i+k-l 

denote  the  mean  squared  error  induced  by  the  optimal  at  mean  squared  estimate  of 


pill  i+k— 1 

^o’  observation  vector  y^  .  Then,  via  straight  forward  computa¬ 
tions,  and  for  c  as  in  (17)  and  as  in  (19),  we  obtain. 


■  i  <^>]  «o) 

■  n,t+k-i«>  +  (■»(-=; n+c')  -  cko)  (21) 


5.  Construction  of  Filtering  and  Smoothing  Operations  -  Step  2 

For  the  same  model  as  in  section  4,  we  now  consider  the  case  where  the 
length  of  the  observation  sequence  is  larger  than  the  integer  m  in  (3).  Consider¬ 
ing  observation  sequences  y^  ;  we  will  distinguish  here  between  causal  filtering  and 

noncausal  filtering  or  smoothing.  In  the  former,  the  information  datum  is 

k. 

estimated,  given  the  observation  sequence  y^  .  In  the  latter,  the  datum  is 
estimated,  when  the  sequence  y^*^  ^  is  observed. 

5.1.  Causal  Filtering 
Let  us  define, 

"k  -  "IWl] 


II  =  R  (l,k)  +R^  (l.k) 


(22) 
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Then,  for  k  _<  m  ,  and  directly  from  lemma  1,  we  conclude  that  the 
Ic 

estimate  (y^^)  is  as  follows. 


^  A  ^ 

X  =  X, 

k  k 


i>'i )  ■  *F,k  K  >'i)  •  ‘‘i" 


(23) 


;  where. 


:  1^1  1  \ 


®F,k 


Xj^sgn  z  ;  |zl  > 


(24) 


,  1/2 
\  "  ^k 


(25) 


c  :  <t>  (c)  +  c  ^  0(c)  =  ^  ^ 


2(l-e) 


(26) 


Let  us  now  assume  that  k>m,  and  let  us  consider  the  estimation  of  the  datum 

I  £  j  1- 

given  the  estimates  j  x^  (y^^)  5  1£  k-l|  ,  and  given  the  observation  sequence  y^. 

"'k— ,''k— ^.T  -k  . 

For  Xj^  :  (x^^  )  =  [x^^ , .  ,  ,Xj^_^] ,  k--£  ^  1,  let  us  define, 


;  1  <  ^  <  k-1 


;  where, 


X  =  „  R-1  x*^-” 

l£  £,'C+m-k  s,k-m  1 

^  „-l  ^k-m 

x  =  G  R  ,  X, 

Ik  km  s,k-m  1 


'"L-  . 


(27) 


(28) 


'^s.k-^  =  (29) 

Given  (:  1  <  i  <  k-1,  the  effective  observation  datum  corresponding  to  the  time  index 
i  is  then,  .  In  addition,  given  the  sequence  of  estimates. 


(y^)  ;  1<  C  1  k-m|  ,  an 


initial  estimate  of  the  datum  x  is  provided  by  x 


Ik 


lor  the  final  estimate  of  the  datum  x  ;  k  •  m,  we  usi'  the  operation 


I  ijj  ^  ^  (  24  )  ,  in  e  on  j  line  t  i  on  with  the  above  e  I  f  ee  t  i  v  i'  observations,  to  ob  t  <i 


V  H..I  v.v  nrF'V'^.K  -.■  . 
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''  A  ^  ,  k. 


T  -1  ^k 

^L  R  ,  X, 

km  s ,  k-m  1 


^k-m  .  , ,  ^ 

’'l  ^Ik’ 


■’^k-m+1  ^I.k-m+l’®’ •  " 

;  k  >  m  (30) 


;  where  \.Rg  and  are  respectively  as  in  (28).  (29),  and  (22),  and  where. 


(■")'■  [v  -.„  ] . 


As  an  alternative,  we  also  consider  the  following  causal  filtering 


operation. 


’.kK  >'S‘)’  ''i  “ 


“L  \  ;  k  >  » 


:whMe,  ’’Y  .  . =  [E{X„\>....E{X^_jX^)],  and, 


I  ~  ®F,m  l^m  ^£-n 


i) 


5 ’ 2  Noncausal  Filtering  or  Smoothing 
Let  us  define. 


(k-l,  k+£)  +  R^(k-£,k+£) 


^kl  ~  '^kl  ^kt 


For  2£+l<m,  and  directly  from  lemma  1,  we  then  conclude, 

-  A  „  /  k+£\  /  T  b+P  \ 

^k(  (^k-fr  ^S.k.l'  {^kC  ^k-f  )  • 


12 


;  where  c 

is 

as  in  (26),  and. 

I 

1 2  ;  Ul 

-  ^k£ 

=  1 

1  .  . 

1 

[  sgn(z)  ;  |z| 

1/2 

-  ^ 

(36) 


(37) 


For  t  such  that  2£  >  m  -  1,  and  m  =  2n  +  1 ,  we  consider  the  following 
smoothing  operation. 


\a 


®S,k,k-l  (\.k-iyi*"  J  2(k-l)<m-l  =2n 


(38) 


T  1 

\,k-n-l  \,k,k-n-l  ^k~m  »  ^  “  2n 


;  where, 


'Vm>'  =  f  <  «nVl>"  .  (  I 


'  [”ii+l,ll . Vl.nj 

("k«"  0  ■["kM.n'-'-  «2k-n-l.„] 

(■’!,„  y^::) 


(39) 


6*  Asymptotic  Properties 

In  this  section,  we  are  focusing  on  the  asymptotic  properties  of  the 
operations  presented  in  section  5.  We  begin  by  proving  that  the  latter 
operations  are  outlier  resistant,  for  asvmp to t i c,i  1 1 v  long  data  sequences. 

For  finite  length  such  sequences,  the  operations  are  outlier  resistant,  since 
they  satisfy  conditi..n  A  in  section  2.  Regarding  asvnqootio  outlier  resistance. 


we  will  go  directly  to  the  definition  of  outlier  resistance,  stated  in  section  2. 
We  will  consider  sequences  of  i.i.d.  m-size  blocks  of  outlier  values,  that  are 
additive  to  and  Independent  of  the  nominal  Gaussian  noise  process.  We  then 
express  the  following  theorems,  whose  proofs  are  in  appendix  B. 

Theorem  2 

Let  the  Gaussian  information  process  in  sections  4  and  5  be  such  that: 

There  exists  some  finite  positive  constant  c,  such  that. 


E{x^}  <c  ;  Vk 

l\,k-l  \,k,k-l  ^2(k-l)l  5  c  ;•  Vk 


l^k  ^s!k-l  Vll  I'"  ‘ 


Vk 


(40) 


T  A  T 

; where  I  =  [1,...,1],  with  n  elements,  and  where  G,  and  R  ,  , 
n  k  s,k-l 

are  respectively  given  by  (28)  and  (29). 

Then,  the  operation  in  (30)  is  asymptotically  (k  <»)  outlier  resistant, 

within  the  class  of  stationary  and  ergodic  observation  processes,  and  for  mutually 

independent  m-size  batches  of  ouliers. 

Theorem  3 

Let  the  Gaussian  information  process  satisfy  conditions  (40)  in  theorem  2. 
Then,  for  mutually  independent  m-size  batches  of  outliers,  the  operations  in  (32) 
and  (39)  are  outlier  resistant,  for  every  k. 


We  will  now  turn  to  the  evaluation  of  the  asymptotic  at  the  nominal  Gaussian 
model  mean  squared  error,  Induced  by  the  operations  in  (32) , (38) , . afid  (30), 
in  the  case  where  the  Gaussian  nominal  information  and  noise  processes  are  both 
stationary,  with  respective  power  spectral  densities,  f  (co)  and  f,,((.))  ,  ..r  [ -tt  ,  tt  ] . 
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We  will  start  with  the  operations  in  (32)  and  (38),  whose  asymptotic  forms,  when 
the  information  datum  Xq  Is  estimated  from  infinite  past  and  future  observation 
data,  are  then  respectively  as  follows. 


‘of  H  ®F,m  ^^m 

£=_«.  £-m+l 


(41) 


/_T  £+n. 

’‘os  =  E  8s,£.n^^£,nW 

£=_oo 

£^0 


(42) 


The  sets  and  {a^}  are  respectively  the  prediction  and  interpolation  coefficients 

corresponding  to  the  Gaussian  nominal  information  process.  The  functions  g„  (.) 

F,m 

and  Se  0  (.)  are  respectively  as  in  (24)  and  (36),  and  they  both  induce  stationary 

processes,  when  operating  on  sequences  from  the  stationary  nominal  Gaussian  model. 

Let  us  then  denote  by  f  (w)  and  f  (w)  ,  we  the  power  spectral  densities 

*F,m  ®S,n 

induced  respectively  by  g_  (.)  and  g„  »  (.)  and  the  stationary  Gaussian  nominal 

r  ytn  O  ytl 

model,  where  then  g„  a  (.)  is  not  a  function  of  t.  Let  D(a))  and  A(a))  ,  we  [-tt,tt], 

o  >  ^  f  n 

be  the  Fourier  transforms  of  respectively  the  sequences  {d^;-<»<£<-l}  and 
{a£;-°®<£<“>,£40}  >  and  let  us  define. 


(cj)  =  [l,e 

r  ,m 


m 


(43) 


b ,  n  n 


(44) 


;  where  P  in  (43)  is  as  in  (22),  and  where  P  in  (44)  is  as  P,  in  (34),  for  any  k 
m  n  kn 

in  the  case  of  the  stationary  Gaussian  nominal  model.  Given  all  the  above,  and 

denoting  by  e(f  ,  x^„)  and  e(f  ,  x_  )  the  mean  squared  errors  induced  respectively  by 
o  U  F  o  US 

the  estimates  in  (41)  and  (42),  at  the  nominal  stationary  Gaussian  observation  density 


f  ,  we  obtain, 
o 


e(fo.  XoF>  ■ 


(U))du>  -  2[2<I'(c)-l]  /  Re(D(w)H„  (aj))f  (a))da) 


-n 


:)-l]  /  R 
J  -ir 


F,m'  "  s 


L 


(oj)  I  |D(a))  II  dw 


(45) 


' 


(a))du)  -  2[2<l>(c) 


-TT 


Re(A{a))H^  (w))f  (u)du 
o  y  n  s 


-7T 


(w)  I  |a(w)  I  I  du) 


(46) 


;  where  Re(.)  denotes  real  part,  where  (|h(to)||  =  h((i))h*((jj) ,  where  ^(x)  is  the 

Gaussian  distribution  at  the  point  x,  and  where  the  constant  c  is  as  in  (26) . 

The  quantities  that  present  problems  in  the  evaluation  of  the  asymptotic  mean 


squared  errors,  in  (45)  and  (46),  are  the  power  spectral  densities  f  (w)  and 


F,m 


f  (oj)  .  These  power  spectral  densities  correspond  respectively  to  the  sequences, 


’S,n 


(Z.)}  and  {g„  (E.)}  of  random  variables,  where  Z.  =  and  E  = 

F,m  1  '’S,n  i  i  m  i-m+1  i  n  i-n 


and  where  for  c  as  in  (26),  X  as  in  (25),  and  y  =  y,  ;  Vk  as  in  (37),  the  functions 

m  n  k.n 


g_  (z)  and  g„  (z)  are  as  follows. 
A*  y  ni  O  y  o 


’F,m 


( 

(z)  =  { 


|z|<X 


m 


(47) 


X  sgn(z) ;  z  PX 
m  m 


go  _(z)  =  ' 

o ,  n 


z;  |z|<y 


y^sgn(z)  ; 


(48) 


We  will  seek  upper  and  lower  bounds  on  the  errors  in  (45)  and  (46) ,  via  the 
derivation  of  such  bounds  on  the  spectral  characteristics  of  the  sequences 


..n.l  Ik,  . 


We  note  that  tlio  power  spectral  densities. 
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T  i 

and  f  (uj) ,  of  respectively  the  Gaussian  sequences  {Z.  =  P  and 

t  1  m  i— m+l 

{E^  =  ^i-n^  *  when  is  generated  by  the  Gaussian  stationary  nominal 

density  are  easily  found  to  be  given  by  expressions  ('^9)  and  (50)  below,  where 

H  (oj)  and  H  (w)  are  respectively  as  in  (A3)  and  (AA), 
r  9  ni  o  y  n 


=  1  |Hp^^(a))  I  I  ^[fg(w)  +  fj^(aj)  )  ;a)e[-TT,Ti] 


(A9) 


f_((j)  =  ||h  „(w)||  [f^(a))  +  f„((i))  ]  ;tJC[-TT,TT] 


S.n 


(50) 


Let  us  now  define  the  following  quantities. 


r  _  =  E{Z  Z  |f  }  ,  r  =  E{E.E..  |f  } 

pF  1  i+p '  o  pS  1  i+p '  o 


2  A 

’^OF  ’  ^ 


A  IeF 


pF  ^2  ‘  “pS 

F 


2  A  2  , 


*^05  ’  ^pS  2  ’  "pS 


2  A  2  ,,  2  , 


(51) 


=  >^pA 


S 


+  2U  (n-2)r 


_2<Kc)-l 
2^  k! 


S  -1 


-k  ^(2k-l) 


(  uVj'-'' 

pA 


)• 


^  Z 


2^-1  „k-^+l 
c  2 


£=l 


k: 


(2k+l)[(i-l)l 


-  2 


-2(k-^+l)  (£-1) 


k:(2^-l) 


f 

):  . 


;A  =  F  or  S 


(52) 


U(x)  = 


j  1  ;  X  >  0 

I  n  ;  X  0 
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K  (  I  =  [24)(c)-ll^f  (w)  + 

N ,  t  n  )  ,  A  s 

P  |<|p|<N-l 


I  ,  i 


-  |24.(c)-l]‘rp^, 


+  o?  {20(c)  (3-c:^-2«(c)]-2c*(c)  +  2c^-2)  ;  A-F  or  s 
A 


(53) 


f. 


4,{n  },F^^o’V  "  j  ~  2[24>(c)-1]  j  Re(D(a))Hp^^(a)))f^(a))du 

^  -71  -77 


/ 


+  /  ||D(a))|i  j^p(a))du) 

P 


(54) 


-77 


4.{np}.S^^o*V  =  ^277)“^!  j 


/■ 

J~n  P  / 


f  (a))da)  -  2[2$(c)-l)  /  Re(A((i))H  (to) )  f  (to)dto 

s  /  S,n  s 

77 


(55) 


;  where  D(a))  and  A((o)  are  respectively  as  in  (45)  and  (46),  where  ^(‘**) 


H-  (to)  are  respectively  as  in  (43)  and  (44),  and  where  c  is  as  in  (26).  Then, 

O  1 1\ 


for  e(f  ,x„„)  and  e(f  ,x_  )  respectively  as  in  (45)  and  (46),  we  can  express  the 
o  Ur  o  US 


following  theorem,  whose  proof  is  in  appendix  B. 
Theorem  4 


Let  r  _  and  r  „  in  (51)  be  such  that,  ri  t  -  ^  1  1,1  0  and 

pF  pS  ’  |pj,F  |p(+l,F 


-1 


r,  ,  c  >  r.  „  \0.  Let  y  and  y  in(59)  be  such  that,  y  „  <  e(l+e)  and 
|p|,S  1p1+1,S  '  'pF  pS  pF 


-1 


Y  „  <  e(l+e)  ,  V  |p|  >  1,  Then,  given  6  >  0,  there  exist  positive  finite  integers, 
ps 


N  and  N  ,  and  sets  of  positive  integers,  {n  ;  1  5.  |p|  f.  ^  -1}  and 


pF 


{"pS  ;  1  1  1p|1  "1p|,f  ^  "1p|+i,f  "|p|,  s  "  "|p|+i,s  ’  P’ 


and , 


.{n  },F^^o’^0^ 

F  pF 


~  V.{n 

S  pS 


;  where  e 


■  r  I  E-d  x„)  and  e 
Np.tn^pl.F  o,  0 


,,  r  1  ^(f  ,x„)  are  given  respectively  by  (54)  and 
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(55) .  The  latter  errors  are  then  used  as  close  approximations  of  respectively  the 

mean  squared  errors  e(f  e(f  ) ,  in  (^*5)  and  (46). 

o  Ur  o  Ui> 


Let  us  now  turn  to  the  estimate  induced  by  the  operation  in  (30). 

For  stationary  information  and  noise  processes,  and  for  the  datum  Xq  being  estimated 
from  infinite  past  and  future  observation  data,  the  asymptotic  form  of  this 
estimate  is  as  follows. 

— m  /  0  — m  ^  V 

^OF  "  S  ®F,m  (  "  53  ‘^j.m+i^jF^  |  (56) 

£=-oo  i=-m+l 

;  where  {^^pl  the  prediction  coefficients  corresponding  to  the  Gaussian 

nominal  information  process,  for  predicting  x^  from  {x^  ;-oo<£<^-p}  ,and  where  {b^}  are 
the  filtering  coefficients  at  the  nominal  Gaussian  model,  for  estimating  x^  from 
{y^  .  The  function  ^(.)  is  given  by  (24),  with  k=m. 

We  will  study  the  asymptotic  performance  of  the  prediction  estimate  in 
(56),  for  autoregressive  Gaussian  nominal  information  processes,  and  additive  white 
Gaussian  nominal  noise  processes.  Let  the  nominal  model  be  described  as  follows, 
u  =  Au  +  Bv 


-n  -  ,  -  , 

n-1  n-1 

T 

y  *  B  u  +  w 

'^n  - 


(57) 


n  n 

;  where  the  sequences  and  {W^}  are  mutually  independent,  i.i.d.  and  zero  mean 

2  2 

Gaussian,  with  respective  variances  o  and  r  ,  and  where. 


f"n’  Vl .  Vk+1^ 


1  ,  0,  ....  0 
0  ,  1,  0,  ....  0 
b . 0,  1 


(58) 


8^  =  11,  0....  01 

ilien,  the  estimate  in  (  5fi)  takes  the  followine,  rorin. 


.Vi' 


SOF  ■  * 


S-m.F  +  ip.mlE  ^iC 


nT  *ni+i 
y,  -  B  A 
1 


“-m.pJ  / 


;  where  sre  the  vector  coefficients  of  the  optimal  at  the  nominal  model 

linear  mean  squared  estimate  of  given  {y^  ;  i  £  O) ,  and  where  for  c  as  in  (17) 

and  for  r  .  being  the  variance  gain  induced  by  the  optimal  at  the  nominal  model 

-J  .m 

linear  mean  squared  estimate  of  >  given  {y^  ;  -m+1  £  i  £  OK  we  have. 


®F,-j ,m^*^ 


X  ;  X 


^  -i  m  ^  ^-1  m 

-J  ,m  -J.nl 


-j  ,m  =  c  [r  . 

J  -J.m 


Let  us  now  define, 


i=-m+l 


—  -m  J 


S  =  E 
g 


S  =  E 
u 


{[  «  0  -a”  2  .„]  [  V  0 


u  -a"*  U 
-  0  -  -m  J  ) 


S  =  E 
uz 


{?  [  "  0  !i  -n,]M  ■  t  'ir  ‘■J-’---''} 


S  ..  =  K  {  (II 

r\  P 


0  Oi' 


0  "oi- 


p=S  -KS  -S  K+S  , 

11  117.  U7  R 


If  ,  :  f  ,  [  '  ,  l)-l  .  ’ 

ij  11  ii  -i  +  l»ni  I] 


Z  h' 


^pip.a|«i«P« 
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li  ■  '^(®F,l-i,m'^''’F,l-1.».''’-)'l?  -SJ  I 


Nl.  S  [2  -1,  .J  s;‘  C-II  * 

.  (  '^-i+1  ,in  \  ,  ,  T  ,  T  -1 

e.  e.  S^  C  -I] 

(g  +  G  -  C..\  C-  C^N..  -N^. 

\  1.1  11  ij/  2  IT  11 


+  [2 


A  -1  T  -1 

Q.,  =  2  ic'  s  ^ 

Ij  2 


cl 


1 

'  '*'^^1'®  i>j  “  l,...,k,  where  e^  =  (0, .  .0,1,0, ..  0)  ,  and  where  the  exnectations 
are  taken  at  the  nominal  model.  We  then  express  the  following  theorem,  whose  proo*" 
is  in  ay-pendix  B. 

Theorem  5 

Let  the  process  { U  }  be  asymptotically  stationary,  and  let  and  be  the 

OF  OF 


solutions  of  the  following  matrix  equations. 


^OF  ■  S^F  '' 

^oF  ■  *"  ^oF  -  0 


(h?) 


;  where,  Q  ={  tr(A"‘  [A^]™  Q.^)  ,  i,j  =  l,...k} 

Then, 

£.  u 

(i)  The  solutions  S  ^  and  S  ^  exist- 

OF  oF 

\ii)  The  residual  process  is  asymptotically  stationary. 

If  means  that  each  diagonal  element  of  the  matrix  is  bounded 

from  above  by  the  corresponding  diagonal  element  of  the  matrix  M^,  then. 


oF 


^  ^oF 


(hi) 


(iii;  if  tr  denotes  trari?,  ami  if  i:  (A)  is  the  larg.fst  m.ignitmU'  e  i  geiiv.i  1  >ii 

max 


ot  thi’  m.Jtrix  A,  then. 


If  the  nominal  model  in  (57)  is  first  order  autoregressive,  with 
autoregressive  parameter  a  <  1,  then  the  matrices  in  (61)  and  (62)  reduce  to 


scalars,  and  the  bounds  in  (63)  are  easily  found  to  be  as  follows , where  c  is  as  in  (17) 


oF 


=  p  [1  - 


2ra  ,-l 
a  q] 


2ra,-l 


(65) 


oF 


=  P  [1  -  a] 


;  where. 


for  =  E{Vq}  and  =  e{Wq}  in  (57) , 


X  ~  X  =  c  fr — 

m  0,m  ^  0,r 

^  E  i[  X  -  a"  X  ]  ^}  “  (1-<X^)"V(1^^“) 
u  (  u  — m  / 


,ra 


2  A 


sii  =  E 


{ [  E  b  (Y  -  X  )  ]  ^  }  5:  b^  +  —  {  S 

i=-'in+l  i=-m+l  1-a  i=-i 


Z  b.b  a 
m+1  j=-nH-l  ^  ^ 


-  H  '  *0  -  o”  I  i:  ,*>1  «i  - 

l=-ra+l 

»  (^)  - 


f  ^  2 


m+i  „  . 

a  X  ) 
-m 


)  0 

r.  ‘’i“"' 


2m  j 
-a  d 


i=-m+l 


11 


(66) 


d  b^ 


A  ?  7  ?  0  ^ 

p  ='  +  s^  +  (sf.-X^  -  2s  )  f  -2X  s,,  4,  (^  ) 

m  u  11m  uz 


,2  -1 


q  =  1  -  d(2-d)f  -  2d^  sT;  X  [1  -  s  s:h  4)(-!5  ) 

IJ.  m  tiT  ' 
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The  scalars  p  and  tr  P,  respectively  in  (67)  and  (64),  are  both  bounded  for 

every  value  of  the  truncation  parameters  {X.  }.  The  bounds  in  (64)  and  (67)  are 

J  ,m 

thus  exponentially  decreasing  to  zero,  with  increasing  m.  The  lower  and  upper 
bounds  in  (63)  and  (65)  are  therefore  approaching  each  other  with  exponential 
rate,  as  the  design  parameter  m  increases. 

7.  Performance  Measures  for  Outlier  Resistance 

Let  us  consider  the  frequently  observed  in  practice  case  of  indeoendent  and 

additive  outliers.  In  particular,  let  the  noise  sequence  W  W^,  ,  .  . . } 

be  such  that  each  of  its  elements  is  generated  by  the  nominal  Gaussian  noise  process, 

with  probability  l-<5,  and  it  is  instead  equal  to  some  deterministic  value,  v,  with 

probability  6,  0  £  6  £  1.  Let  the  value  v  occur  with  probability  6,  independently 

per  noise  datura.  Given  the  above  outlier  model,  given  some  asymptotic  filtering  or 

smoothing  operation,  x^,  let  e(f^,6,v,XQ)  denote  the  induced  mean  squared  error. 

That  is,  if  f  represents  the  overall  nominal  Gaussian  model,  then,  e(f  ,6,v,x„)  = 
o  o  U 

=  E{[X„-x_]^|f  ,6,v}.  Let  us  denote,  e(f  ,6,x„)  =  lim  e(f  ,6,v,x„),  and  let  there 
u  u  o  o  U  .  o  U 

v->4<» 

exist  some  value  6*,  0  _<  <5*  <  1,  such  that. 


e(f  ,6,5-)>  Et'xjlf  }  ;  V6  >  5* 
o  U  0  o 

e(f  ,6,5  )  <E{X^|f  }  ;  V6  <  6* 
o  0  —  0  o  - 

★ 

Then,  the  value  6  is  called  the  breakdown  point  of  the  asymptotic  operation  x^. 
The  breakdown  point  clearly  represents  the  maximum  frequency  of  independent. 
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We  note  that  the  breakdown  points  of  the  optimal  at  the  Gaussian  models  linear 
filtering  and  smoothing  operations,  are  easily  found  to  equal  zero . 

Let  us  now  consider  a  generalization  of  the  outlier  model  presented 
above.  In  particular,  let  us  consider  the  case  where  independent,  size  m 
blocks  of  outliers  may  occur.  Then,  each  block  occurs  with  probability  6,  and  it 
consists  of  a  value  v  per  datum  in  the  block.  Given  some  filtering  or  smoothing 
operation  x^,  we  then  denote  the  induced  mean  squared  error,  e^(f^, 6,v,Xq) .  Denoting 


by  e(f^,XQ)  the  mean  squared  error  in  the  absence  of  the  above  outlier  model,  we 
denote,  J  xCv)  =  »v,X-)  -  e(f  ,x_)  .  We  call  J  j.(v)  the  variation 

function  at  6.  Given  6,  the  variation  function  exhibits  the  difference  between 
the  mean  squared  error,  when  the  outlier  value  is  v  and  the  frequency  of  the 
outlier  blocks  is  6,  and  the  mean  squared  error  in  the  absence  of  outliers. 

We  call  I_  ^(v)  =  6  x(v),  the  normalized  variation  function  at  6.  and  we  call 

m,u  m,*-)  - — — - 

I_  (v)  =  lim  I_  ^(v)  the  influence  function.  The  influence  function  is  the  sIod 

(S-O  - 

of  the  variation  function  at  ^  =  0,  and  it  exhibits  the  effect  of  the  outlier  value 
V,  at  asymptotically  small  outlier  frequencies  6.  It  is  easily  found  that  the  optimal 
at  the  Gaussian  model  linear  filtering  and  smoothing  operations  induce,  for  m=l, 
influence  function,  I^(v),  that  is  given  by  the  following  expression,  where  C(ci))  is  the 
Fourier  transform  of  the  linear  filter  or  smoother,  and  where  fjg(w)  denotes  the  power 
spectral  density  of  the  Gaussian  noise. 


Ij(v)  =  (2n 


I  Iv^  +  (27t)  '  j  fj^((a)da.>  1  j 


TT  ^  tt 

(w)da'  1  /  II  i  I  ‘"df. 

-Tl 


-  2 


/ 


iC(u))  11  fj^(Gj)d(.l 


(68) 


As  .1  f  vine  Cion  of  the  on  l  1  i  or  va  1  lu-  v  ,  t  lit'  i  n  t  1  mnie  o  f  line  t  i  c>n  I  ^  (  v )  is  I'jii  .id  r  a  t  i  e  , 
arui  it  irureases  [n  intinifv  as  v  ►  + 
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In  this  section,  we  study  the  breakdown  points  and  the  influence  functions 

of  the  operations  in  (32)  and  (38),  and  the  operation  in  (30).  We  adopt  respectively 

the  same  nominal  Gaussian  models  as  in  section  6,  and  we  start  with  the  operations 

in  (32)  and  (38).  Let  us  then  define  the  spectral  densities  f  ((j)  ,  f  (w)  ,  and 

®  %,n 

f  (cj) ,  and  the  Fourier  transforms  A(a))  and  D(w)  as  in  section  6.  Let  H  (w) 

^F,m 

and  Hg  (^3)  and  (44),  and  let  us  define. 


r  (k)  =  (2tt)  W  f  (o))  e  ^‘^''dLi 
®F,m  •'-TT  ^F,m 


r  (k)  =  (2tt)  M  f  (cu)  e 
,  n  •'-TT  ,  n 


U(x)  = 


1  ;  X  _>  0 
0  ;  X  <  0 


Then,  after  some  straightforward  transformations,  we  find  the  following 
mean  squared  expressions,  for  c  as  in  (17). 


e(f  .6,x  )  =  (27t)  ^  /  f  (u))doj  +  A^D^(O)  - 

o  OF  Is  m 

J--n 


(1-6)'"!  2[2<J>(c)-1]  (2^1)  /  Re(D(u))H,,  (w))f  (a))dw 

/  F,m  s 


+  2A^D^(0)  -  [A"  +  r  (0)  j  (?7t)'^  / 
m  ni  R,-  / 

i,m  ■'-Ti 


I  D((i.))  I  I  dw 


m-1  -y  f  ^ 

+  U(m-2)  V  (l-6)'"^^[r  (k)  +  A^1(2tt)“M  1|d 

11  Ri-  m  / 

k=l  F,m  J-v 


(a.)|!2(e-i'"'^  +e-i‘"‘^ldai 


+  (1-M^"'(2r)  ’lA^D^O)  -A^  /  |  !  IKo.)  |  |  ^  (  ^  e^'‘’''ldi.  + 

k=-m+l 


I'(  .  )  ■■  t  I  d  . 


iU  .  )  i  r  1  ld>.  }  (  70) 

k-^inil'-”' 
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e(fo,6,Xos)  =  (2Tf)  ^ 


A^(0)  - 


(1-6)  j  2[2<I>(c)-1}(27t)  ^  /  R  (A(cj)H  (oj) )  f  (to)d(iO 


/p  n 

+  2uy(0)  -  +  (0)](2tt)-^  /  ||A(aj)l|2duj  I 

•'-IT 


+  U(n-l)  (k)  + 


r  TT 


+  ^y(O)  -  u^f  ||A(u)||^( 

-TT 

+  f  ||A(w)||2f  (a))da)  -  /  j  j 

-^_Tr  ®S.n  J-Tj 


^  eJ^'^ldo)  + 

k=-2n 


(k)  e3“'‘)dd,  ) 

k— 2n®‘'' 


;  where  end  are  respectively  as  In  (47)  and  (48).  Let  us  now  consider 
Che  nuentities  {b^^(n)),  in  (52).  and  ,_^(u).  in  (53),  and  let  us  denote. 


“a(“)  ' 


C^(ui) 


1  r  ,in 

;  for  A=F 

(h„  (cu) 

o  y  n 

;  for  A=S 

> 

£ 

;  for  A=S 

Id(u)) 

;  for  A=F 

/i  2 

;  for  A=F 

(  2 

U 

n 

;  for  A=S 

in  ;  for  A=F 


2n+l  ;  for  A~S 


•>  •  •  •_»  Ji'  /  .*•  !.*•  **•  •>  .  -’V«^ 


Let  us  then  define. 


/TT 

Re(C^(oj)H^(oj))fg(w)dw 

T 


+  2v^C^(0)  -  +  bQ^Cn^)  ]  (2tt)  ^  /  |  |  C^((J)  |  |  ^do)  ( 


^  n  /*  ^ 

U(n^-2)  X;  (1-6)  ^  +  bj^(nj^)l(2Ti)“^  I  |  | C^(lo)  ]  1  ^ [e^^^V  e“^‘^‘"]du) 

^=1  J-v 


/■n  n^-1 

l|C.(a))!|^I  X)  e^'^^'ldo)  + 
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TT  A 


/7T  ^  IT 

-IT  ^  -TT  A 


;  A  =  F  or  S 


We  now  express  the  following  theorem,  whose  proof  is  in  appendix  C. 


Theorem  6 

*  jc* 

Let  the  conditions  in  Theorem  4  be  satisfied.  Let  6_  and  6  denote 

F,m  S,n 

then  the  breakdown  points  of  respectively  the  filtering  and  smoothing  operations 

in  (32)  and  (38).  Then,  given  i;  >  0,  there  exist  positive  finite  integers 

N  and  N„ ,  and  sets  of  positive  integers,  {n  ;  1  <  |p|  <  N  -l}  and 
F  S  pr  —  —  r 

{npg  ;  1  5  ip|  <  Ng-1},  such  that,  njp|^p>  njp,  and  n|p|^g>  nj^i  V  p, 

and  such  that  when  substituted  in  respectively  F  (6)  and  F  (6),  they  give, 

r  o 

(i)  Unique  roots  of  the  functions  Fp(6)  and  Fg(6),  respectively  denoted 

6°  and  6° 

I ,m  S  ,n 


1-5;  -  6"  I  w, 

i ,m  F,m 


16  -  6"  I 

'  S.ii  S.n  ' 


27  - 


it  * 

The  breakdown  points  6_  and  6^  are  unique,  and  they  are  both  strictly 

r  .m  S «n 


larger  than  zero. 


We  point  out  that  the  filtering  and  smoothing  operations  in  (32)  and  (38) 
induce  bounded,  for  every  value  v,  variation  and  influence  functions.  We  do  not 
include  the  computation  of  bounds  on  the  latter  functions,  due  to  the  tediousness 
in  their  derivation.  Wc  now  proceed  with  the  study  of  the  breakdown  point  and  the 


influence  function  of  the  filtering  operation  in  (30),  for  the  nominal  model  in  (57) 

.0, 


For  the  latter  model,  we  first  compute  the  influence  function,  I  (v) ,  induced  by  the 

m 

optimal  at  the  nominal  model  filtering  operation.  Then,  we  study  the  breakdown  point 
and  the  influence  function,  I  (v) ,  induced  by  the  outlier  resistant  filtering  opera- 


m 


tion  in  (30).  Let  C  be  the  kxk  matrix  defined  in  (61),  and  let  us  define, 
.  0 


y  =  E  b.  =  {  U  } 
i=-nri-l'^ 


(74) 


0 


N  =  I  b.  ^ 

i=-m+r" 


Then,  we  can  express  the  following  lemma,  where  for  the  nominal  model  in  (57), 


2  A  2 

r  =  e{Wq}.  The  proof  of  the  lemma  is  in  appendix  C. 


Lemma  2 


Let  the  nominal  model  in  (57)  be  asymptotically  stationary.  Then,  the  influence 


function  I  (v)  is  given  asymptotically  by  the  following  expression,  where  I  denotes 
m 


the  kxk  identity  matrix. 


I°(v)  =  Z  [  (I-C)  [v^  li  /  -  r^N]  [  (a”^)"  (I-C)'^]' 

i=0 


(75) 


We  note  that  for  the  scalar  form  of  the  model  in  (57),  with  autoregressive 


parameter  a  <  1,  the  Influence  function  in  (75)  reduces  to  the  following  expression. 

(76) 


m 


2  2  2 
V  u  -  r  V 


2m  J..2 

1  -  a  (1  -  fj 


r^-ms^  r\^-  lA  -iAM  W  ■ 


-  28  - 


;  where, 


A  ^  2 

V  =  2.  bi 

i=-m+l 

A  ^ 

5-E  “lO 

i=-in+l 


(77) 


>■ 

r 

A 

V 

.A 
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) . 


A  0 

v^  =  E  b, 

i=-m+l 

Let  us  consider  the  quantities  defined  in  (61)  and  the  vector  ]J  defined  in  (74), 
and  let  us  in  addition  define, 

=  E  b  B^[U  -  a'^^u  ]  =  [Zn’*-‘2ikJ 

^  i^-m4l  ^  ^  -m  11  Ik 

S  =  E{Z  Z^}  ={p^  ;  i.j=l,..,k} 

Lx  Ij 


■f. 


^,»  ■  *  'li)  4,„«i  +  vw)) 

f  i  ~  *‘-H-l.ni. 

"ll  "ll 

F  =  {f..  :  f..  =  f.  ,  f..  =  0  ;  i+j} 

V  ij,v  ii,v  i,v  ij,v 


(78) 


A  T 
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V  U  V  UZ  UZ  V  g,v 


1J,V  -  «Sr.i.i,„(z,i+vu,)  gp  „(Zjj4vUj)IZ,  Zj-s^^l) 


H..  Spie'^S^  S-‘ 

1J,V  —1  UZ  z 


N..  =  [f.  e.  e^  +f.  e  e^I  [S^  3  ^C-I] 

ij.v  j,v  -j  -1  i,v  -i  -,i  UZ  Zj 

Q..  =  2'^[C^  S“^  (H, .  +  H..  -G..  )  S'^  C-C^  N, ,  -nT.  C] 

ij,v  ^  z^  ij.v  ji.v  ij.v'  z^  ij,v  ij,v 

A 

A  =  (A-  .  A  1 

-TQ  0,m  -k+l,m 


Wf  ('an  then  express  the  following  theorem,  whose  proof  is  in  appendix  C. 
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Theorem  7. 

Let  the  nominal  model  in  (57)  be  asymptotically  stationary,  and  such  that  the 
elements  of  the  matrix  A  are  all  nonnegative.  Then, 

(i)  Let  (v)  and  I^(v)  be  the  respective  solutions  of  the  following  matrix 
m  m 

equations. 

I^(v)  =  +  A  I^(v)  (A  )  -  +  A  S^p(A  )  -S^p 

(79) 

l‘^(v)  =  +  a”  i;;‘(v)  (A^)  -S^  +  A*"  (A^) 

ra  V  m  or  Or 

;  where,  =  {tr(A“  I^(v)  [A^]  Q^^)  f  tr(A‘"  s“p  [A^l  ^)  ;  i,  j=l, . .  ,k} , 

and  where  and  S^„  are  as  in  theorem  5. 
oF  oF 

Then,  the  influence  matrix  function,  I  (v) ,  induced  by  the  filtering  operation 

m 

in  (30)  and  the  nominal  model  in  (57),  is  bounded  from  above  and  below  as  follows. 

tr  I^(v)  <  tr  I  (v)  <  tr  l'*(v)  (80) 

m  —  m  —  m 

£ 

(ii)  Let  S^(6)  and  S  (6)  be  the  respective  solutions  of  the  following  matrix 
m  m 

equations,  given  6:  0  ^  6  £  1. 

S'*(6)  =  a"*  s''(6)(a'^)  +  (l-d)"*  P  +  (l-(l-6)’"l[S„  +  X  aIi 

(81) 

S^(6)  =  A™  S^(6)  (a'^)  -  (l-d)"*  (R-Pl  +  [1-(1-6)"'][S  +  X 

tB  IP  U  — *u  "TP 

;  where  R  =  {tr(A'"  (a'^]  Q..)  ;  i,j=l,..,k}.  Define,  =  E{l^|f  }. 

m  ij  o  ^  o 

Then,  the  functions  f, (6)  =  tr(S^(6)  -S  )  and  f,(6)  =  tr(sf(6)  -S^)  have  unique  roots 

i  in  o  ^  in  o 

£ 

respectively  denoted  6^  and  5^.  Those  roots  are  both  strictly  positive  and  less  than 

mm 

one.  Furthermore,  the  breakdown  point,  of  the  operation  in  (30)  is  unique,  and 

such  that, 

-u 

5  <  5  <  6 

m  -  m  —  m 


(82) 
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In  the  scalar  case,  where  the  nominal  model  in  (57)  is  first  order  autoregressive, 


with  autoregressive  parameter  a:  0  <  a  <  1,  the  results  in  theorem  7  simplify  as  follows. 


/,  2m.  2m  /  ,  2m,  2m,  2m 

p^(l-qa  )  +  pa  p(q^  +  a  )  )  +  P«  P 

„  2  r,  2m.  2m.  —  —  .,2  ,,  2m, ,,  2m, 

(l-qa^“)  )(l-q“  )  (l-a^™)  ^ 


5“  <  6*  <  5^ 
m  —  m  —  m 


6  solution  of 
m 


.  s^  +  \^  +(l-6)*”tp-s^-X^] 
,u,r,  A  u  m  u  m 

f  (5)  =  - — - 

1-a 


6  solution  of 
m 


2  .  \2  rviUr  2  ,2, 

/  A  s  +  X  +(1-6)  [p-s  -X J 

r(6)  =  “  ---° - = — y— 3- 

l-a^“  +  (1-6)  V“(l-q) 


2  ^  2 

;  where  a  =  E{V_}  for  V-  as  In  (57),  where  p,q,s  ,8  ,X  ,  and  c  are  as  In  (66), 
U  U  u  uz  m 


where  p  is  as  in  (77),  and  where,  for  s^^^^  as  in  (66)  and  r^  =  (57), 


vy+X  vp-X 

H - -)  -  ♦( - -) 

P  P 


0  0  -7  7  0 

p  -  s'"  +  f  [p^  +  V  p'^  -X'^  -  2s  ]  +2ppv[(()(— — 5^)  -(J)(— ^)] 
u  V  m  uz  p  p 


vP-X  vP+X  2 

+  p(vp-X^)  ♦(— r^)  -P(vp+X^)  +  X;" 

n  p  IQ  p  ni 

,  X  vp+X  vp-X 

c(2-c)  f  +  c^{  —  [1-  -^1  IH - -)  +  ( - -)] 

V  I  o  D  0 


s  vp+X  vp-X  \ 

pv[<|»( - -)  -  H - ^)]  } 

P  P  ' 


We  conclude  this  section  by  presenting  the  form  of  the  Influence  functions  induced 


by  the  filtering  and  smoothing  operations  in  (32)  and  (38),  when  the  latter  are  modified 


to  operate  on  disjoint  rather  than  sliding  block  data.  Let  then  H_  (u)  and  H.  (u)  be 

F ,m  a ,  n 


as  in  (43)  and  (44),  let  c  be  as  in  (17),  let  X  and  p  be  as  in  (47)  and  (48),  let  A(w) 

m  n 


and  D((jj)  be  as  in  section  6,  let  f  (w)  be  the  power  spectral  density  of  the  stationary 

s 


information  process,  and  let  us  define. 


(86) 


tp  =  E  {[p"^  Xt  |f  } 

F  m  i-m+1  '  o 


tl  =  E  {[P^  If) 


P  (v)  =  E  {[g  (P^  vH  (0))  -  g  vj  )  1^  If} 

r  ,in  r  ,in  m  i— m+i  r,ni  (‘•mm  i— m+l  0 


P-  (v)  ^  E  {(g  (P^  xj;^  +  V  H  (0))  -  g  (P^  If) 


T  i 


;  where  represents  the  nominal  model.  Let  the  nominal  noise  process  be 


stationary,  let  then  h  (w,v)  and  h  (w,v)  be  the  Fourier  transforms  of  the 

8p  ™  8<.  „ 

t  y  in  ^  ^  n 


sequences  {q„  (v) }  and  {q„  (v)},  and  for  Z.  and  E,  as  in  (51)  let  f  (oj) 

F.P  S.p  1  1 


and  f  (to)  be  respectively  the  Fourier  transforms  of  the  sequences 

®S,n 


{E{g„  (Z.)  g_  (Z_  )|f  }}  and  {Efg.  (Ejg_  (E_,,  ...  )}}.  Then,  the  influence 

F,ra  1  ®F,m  i+mp  '  o  S,n  i  S,n  i+(2n+l)p 


functions  of  the  corresponding  filtering  and  smoothing  operations  are  respectively 


given  by  the  following  expressions. 
TT 


J''') 

F,m 


f-i' 


D((i))||^[h  (a),v)  -  f  (w)  +2  ^  p  (v))  do) 

8p  m  8p  „  F,m 

t  f  in  i*  ^  in 


,  ^  -V  H„  (0)  X  +v  H„  (0) 

-TT  [  ( - r - ^ - )  +  4>  ( - r - * - )-  2  <!>  (c))' 


(87) 


S  9  n 


/■' 


L 


Re(D((i))  H„  (u)))  f  (u))du) 
r  ,ra  s 


I  A(u))  I  I  ^  ( h  (w,v)  -f  (w)  +2  ^  p  (v))da) 
®S,n  Ss.n  S’" 


y  +  vH.,  (0) 


U  -  vH„  (0)  ..  .  ... 

TT  [  4>( - 7 - - )  +  '1'  ( - )  -2  (c)] 

S  S 


Ki’(A(ij)  H„  (co))  f  ((.i)d(ii 
S  ,  n  s 


(88) 


In  contrast  to  the  influence  functions  Induced  by  the  optimal  at  the  nominal 
model  linear  filtering  and  smoothing  operations  (see  (68)),  the  functions  in  (87) 
and  (88)  remain  bounded  for  every  v  value.  In  fact,  from  the  latter  expressions  we 
easily  conclude, 


im  I  (v)  =  TT  ^[2  $(c)-l]  / 

.  ^  J 

* _ tr 


Re(D(w)H„  (w))  f  (a))da)  + 
r  ,m  s 


+  [2"^  +  4)  (-C)  -  c 

IQ 


4)(c)]  TT~^y* 


!  D  (w)  I  !  dOJ 


lim  I  (v)  = 

V  -*■  +  00 


Re  (A(aj)  H,.  (w) )  f  (w)du) 

S  ,n  s 


/TT  -TT 

llD(a))||^  f  (w)du) 

=  Tr~^(2  i  (c)-l]  j  Re  (A((u)  Hg^^((u)) 

+  [2"^  +  $  (-C)  -c~^  (J)  (c)]  7t"^/ 

7-71 

-  IT  ^  /  )  |A(a))  1 1^  f  (a))daj 

.  ®S,n 


(A(a))  1 1  ^doj 


|A(a))  1 1  f  (a))daj 

®S,n 


8.  Numerical  Results 


In  this  section,  we  present  some  numerical  results,  regarding  the  performance 
of  the  filtering  and  smoothing  operations  in  section  5.  We  consider  the  nominal 
model  in  (57),  and  we  first  evaluate  the  performance  of  the  causal  filtering  opera¬ 
tion  in  (59).  Defining, 

5^  :  The  solution  of  the  matrix  equation, 

I  =  A(E-ZBB'^i:(B'^EB+r^)  Ia"®^  -Kj^  Bb’^ 

8  =  EB(B^EB  +r^) 

R^  :  The  solution  of  the  matrix  equation, 

T  2  T 
R  =  AR  A  +0  BB 
o  o 

we  coin[uit<‘  tlie  asymptotic  vector  coefficients  (b,),  and  tin  quint  it  ic''  in  (bl),  as 


loll ows , 


-33- 


T 

=  [(1-6  B  )A]  6  ;  i=-nH-l, . .  .  ,0 


ni— 1  'T  *1  T 

C  =  E  B  )A]  6  A 


t=0 

S  =  R  -A™  R  (A^) 
u  o  o 


m 


m-1 


,Tv.,  o  „T,„  ,,T,  ,m-i„  , 


S  =  y,  [(I-B  B^)A]  6  B  (R  (A^)  -A  "  R  (A  )  ] 
uz  -  -  o  o 


;  where  u(i-j) 


S  =  y  [(1-6  b'“’)A]  6  B^[A^(I-6  b'"')]  + 

^  i=0 

E  Z  [(i-Bb'^)a]  6  b'^[a“^^"^^r 

i=0  j=0 

-  a”'^  r^  (a’’^)'"'^]  b  e"^  [a'^(i-6  B*^)  ] 

(i-3  ; 

[  0  ;  otherwise. 


We  studied  quantitative  performance,  for  the  following  two  special  cases  of 
the  model  in  (57). 

Model  1  First  order  autoregressive,  with  autoregressive  parameter, 

2  2 

a  =  0.5,  and  a  =  r  =  1 . 

Model  2  Third  order  autoregressive,  with  a^  =  0.6,  a^  =  0.07,  a^  =  -0.06  in 

(58),  and  =  r^  =  1. 

Tables  1,  2,  and  3,  and  figures  1  and  2,  exhibit  the  performance  of  the  causal 
filtering  operation  in  (59),  for  various  values  of  the  design  parameters  e  and  ra, 
when  the  nominal  model  is  model  1.  When  the  nominal  model  is  instead  model  2,  the 
corresponding  performance  is  exhibited  by  tables  4,5,  and  6,  and  figure  3.  Tables 
2  and  5  correspond  to  independent  per  datura  outliers,  while  tables  3  and  6  correspond 
to  size-ra  independent  batches  of  outliers.  The  above  tables  and  figures  speak  for 
themselves.  The  causal  filtering  operation  in  (59)  can  combine  dose  to  optima] 
at  the  nominal  model  performance,  together  with  excellent  protection  against  outliers. 
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In  addition,  this  operation  is  more  appropriate  for  protection  against  independent 
batches  of  outliers.  Similar  results  are  drawn,  when  the  order  of  the  nominal  model 
in  (57)  is  some  arbitrary  number  k. 

We  studied  the  performance  of  the  nonrecursive  causal  operation  in  (41),  when 
the  nominal  model  is  model  1.  Our  results  are  exhibited  in  tables  7,  8,  and  9, 
and  in  figure  4,  where  various  values  of  the  design  parameters  e  and  m  are  considered. 
Comparing  the  latter  tables  and  figures,  with  tables  1,  2,  and  3,  and  figures  1  and  2, 
we  conclude  that  the  nonrecursive  causal  operation  in  (41)  induces  higher  asymptotic 
mean  squared  error  at  the  nominal  model,  than  the  recursive  operation  in  (59)  does, 
while  the  former  induces  lower  saturation  point  of  the  influence  function.  Considering 
this  tradeoff,  we  claim  that  the  recursive  operation  in  (59)  is  more  appropriate, 
for  the  autoregressive  nominal  model  in  (57) . 

We  evaluated  the  performance  of  the  smoothing  operation  in  (42) ,  for  various 
orders  of  the  autoregressive  model  in  (57).  Our  results  were  similar  to  those 
exhibited  in  tables  1  to  6,  and  figures  1  to  4.  The  smoothing  operation  in  (42) 
is  thus  as  powerful  as  the  filtering  operation  in  (59). 

9.  Conclusions 

We  proposed  and  analyzed  nonlinear  filtering  and  smoothing  operations,  for 
effective  resistance  to  outliers,  and  simultaneously  good  performance  at  the  Gaussian 
nominal  model.  We  note  that  a  filtering  operation,  similar  to  our  recursive  such 
operation  (in  (59)),  was  earlier  considered  by  Masreliez  and  Martin  (1977).  The 
latter  authors  assumed,  however,  that  the  process  formed  by  the  residuals  is 
Gaussian,  and  used  a  covariance  recursion  to  define  their  recursive  filter.  The 
above  assumption  effectively  reduces  the  problem  to  the  class  of  filters,  that  do  not 


involve  nested  nonlinearities. 


m 

- 1 

1 

2 

3 

4 

5 

6 

0.002 

0.53167 

0.66941 

0.53284 

0.56629 

0.53333 

0,54159 

0.53346 

0.53552 

0.53350 

0.53401 

0.53351 

0.53364 

0.01 

0.53488 

0.67108 

0.54136 

0.54945 

0.54183 

0.54385 

0.54195 

0.54246 

0,54198 

0.54211 

0.1 

0.58157 

0.70620 

0.608293 

0.63797 

0.61640 

0.62328 

0.01851 

0.62032 

0.61904 

0.61949 

0.61917 

0.61929 

0.15 

0.60983 

0.72961 

0.64401 

0.67249 

0.65401 

0.66099 

0.65659 

0.65832 

0.65723 

0.65767 

0.65740 

0.65750 

0,25 

0,71376 

0.73998 

0.72608 

0.73249 

0.72921 

0.73080 

0.72999 

0.73039 

0.73019 

0.73028 

0.3 

0.70079 

0.80718 

0.74848 

0.77357 

0.76146 

0.76758 

0.76474 

0.76626 

0.76556 

0.76594 

0.76576 

0,76586 

0.4 

0.76727 

0.86426 

0.81887 

0.84156 

0.83243 

0.83795 

0.83582 

0.83719 

0.83667 

0.83701 

0.83688 

0.83697 

Table  1 


Bounds  on  the  asymptotic  mean  squared  error,  at  the 
nominal  model . 

Model  1.  Causal  filtering  operation  in  (59). 

Asjnnptotic  mean  squared  error  induced  by  the  optimal  at  the 
nominal  model  causal  filter  =  0.53112 
Upper  lines:  lower  bounds. 


m 

1 

2 

3 

4 

5 

6 

0.002 

0.09928 

0.14352 

0,06814 

0.07476 

0.04932 

0.05048 

0.03788 

0.03811 

0.03056 

0.03060 

0.02556 

0.02557 

0.01 

0.14699 

0.20676 

0.10040 

0.10942 

0.05597 

0.05628 

0.04522 

0.04528 

0.03786 

0.03788 

0.1 

0.32204 

0.40878 

0.21602 

0.22978 

0.15715 

0.15974 

0.12180 

0.12228 

0.09898 

0.09908 

0.08326 

0.08328 

0.15 

0.38225 

0.47011 

0.25595 

0.27034 

0.18674 

0.18937 

0.14516 

0.14568 

0.11824 

0.11835 

0.09962 

0.09964 

0.25 

0.48129 

0,56488 

0.32349 

0.33815 

0.23761 

0.24036 

0.18576 

0.18631 

0.15194 

0.15205 

0.12840 

0,12842 

0.3 

0.52466 

0.60450 

0.35423 

0.36875 

0.26119 

0.26395 

0.20477 

0.20533 

0.16783 

0.16795 

0.14203 

0.14206 

0.4 

0.60417 

0.67478 

0.41329 

0.42723 

0.30739 

0.31012 

0.24246 

0.24301 

0.19955 

0.19967 

0,16937 

0.16940 

Table  2 

Bounds  on  the  breakdown  point. 

Model  1.  Causal  filtering  operation  in  (59).  Independent 
per  datura  outliers. 

Upper  lines:  lower  bounds. 


m 

1 

2 

4 

5 

6 

0.002 

0.09928 

0.14352 

0.13164 

0.14394 

0.14080 

0.14394 

0.14315 

0.14394 

0.14315 

0.14394 

0.14389 

0.14394 

0.01 

0.14699 

0.20676 

0.19073 

0.20686 

0.20274 

0.20682 

0.20580^" 

0.20682 

0.20656 

0.20682 

0.20676 

0.20682 

0.1 

0.32204 

0.40878 

0.38537 

0.40676 

0.40125 

0.40653 

0.40520 

0.40651 

0.40618 

0.40651 

0.40643 

0.40651 

0.15 

0.38225 

0.47011 

0.44639 

0.46759 

0.46212 

0.46733 

0.46601 

0.46731 

0.46698 

0.46731 

0.46723 

0.46731 

0.25 

0.48129 

0.56488 

0.54234 

0.56195 

0.55688 

0.56166 

0.56045 

0.56164 

0.56134 

0.56164 

0.56156 

0.56164 

0.3 

0.52466 

0.60450 

0.58298 

0.60152 

0.56672 

0.60124 

0.60010 

0.60121 

0.60093 

0.60121 

0.60114 

0.60121 

0.4 

0.60417 

0.67478 

0.65577 

0.67193 

0.66775 

0.67166 

0.67067 

0.67164 

_ 

0.67140 

0.67164 

0.67158 

0.67164 

Table  3 

Bounds  on  the  breakdovm  point. 

Model  1.  Causal  filtering  operation  in  (59).  Independent 
size-m  batches  of  outliers. 

Upper  lines:  lower  bounds. 


m 

— 

1 

2 

3 

5 

6 

0.002 

0.55402 

0.83214 

0.57594 

0.68407 

0.59937 

0.63361 

0.61040 

0.62154 

0.61445 

0.61764 

0.61566 

0.61658 

0.01 

0.57548 

0.86214 

0.62504 

0.73994 

0.66518 

0.70200 

0.68180 

0,69383 

0.68763 

0.69109 

0.68936 

0.69035 

0.1 

0.62110 

0.89436 

0.69155 

0.79589 

0.72865 

0.76040 

0.74097 

0.75110 

0.74499 

0.74788 

0.74615 

0.74698 

0.15 

0.65204 

0.94013 

0.72942 

0.83110 

0.74120 

0.79568 

0.77011 

0.78320 

0.77401 

0.77516 

0.77432 

0.77501 

0.25 

0.69875 

0.95182 

0.73479 

0.86264 

0.76678 

0.80203 

0.78133 

0.79312 

0.79002 

0.79202 

0.79012 

0.79136 

0.3 

0.73478 

0.96067 

0.73930 

0.91011 

0.78033 

0.86481 

0.79300 

0.82414 

0.80400 

0.80923 

0.80511 

0.80547 

0.4 

0.73510 

0.97033 

0.74902 

0.91437 

0.79087 

0.86690 

0.81142 

0.83571 

0.82267 

0.82610 

0.82320 

0.82359 

Table  4 

Bounds  on  the  asymptotic  mean  squared  error  at  the  nominal 
model . 

Model  2.  Causal  filtering  operation  in  (59).  Asymptotic  mean 
squared  error  induced  by  the  optimal  at  the  nominal  model 
causal  filter  =  0.54731. 

Upper  lines:  lower  bounds. 


1 

2 

3 

4 

5 

6 

0,002 

0.07594 

0.13890 

0.05802 

0.07501 

0.04513 

0.04960 

0.035395 

0.035980 

0,028765 

0.029010 

0,02411 

0.02486 

0.01 

0.11029 

0.20020 

0.08225 

0.11510 

0.06334 

0.08010 

0.04958 

0.05156 

0.04030 

0.0450 

0.03380 

0.03388 

0.1 

0.25689 

0.39537 

0.18640 

0.22540 

0.14353 

0.15003 

0.11313 

0.11804 

0.09248 

0.09424 

0.07790 

0.07823 

BB 

0.32899 

0.47563 

0.23552 

0.27100 

0.18083 

0.20242 

0.14286 

0,14811 

0.11705 

0,11829 

0.09878 

0.09890 

0.25 

0.47094 

0.60225 

0.33123 

0.39693 

0.25350 

0.26089 

0.20121 

0.20541 

0.16563 

0.16735 

0.12747 

0.12784 

0.3 

0.53838 

0.65802 

0.37811 

0.42004 

0.28952 

0.29457 

0.23047 

0.23215 

0.19020 

0.19082 

0.16150 

0.16195 

0.4 

0.66166 

0.75106 

0.47002 

0.52401 

0.36191 

0.39102 

0.29019 

0.30016 

0.24090 

0.24210 

0.20548 

0.20602 

Table  5 

Bounds  on  the  breakdown  point. 

Model  2.  Causal  filtering  operation  in  (59).  Independent 
per  datum  outliers. 

Upper  lines:  lower  bounds. 


m 

1 

2 

3 

4 

5 

6 

0.002 

0.07594 

0.13890 

0.11269 

0.13995 

0.12939 

0.14500 

0.13424 

0.13952 

0.13578 

0.13595 

0.13622 

0.13682 

0.01 

0.11029 

0.20020 

0.15774 

0.19500 

0.17826 

0.19851 

0.18406 

0.18820 

0.18592 

0.18683 

0.18643 

0.18682 

0.1 

0.25689 

0.39537 

0.33813 

0.39220 

0.37173 

0.39104 

0.38136 

0.38804 

0.38444 

0.38740 

0.38530 

0.38607 

0.15 

0.32899 

0.47563 

0.41556 

0.47215 

0.45031 

0.46903 

0.46022 

0.46630 

0.46336 

0.46502 

0.46424 

0,46482 

0.25 

0.47094 

0,60225 

0.55275 

0.60112 

0.58401 

0.60039 

0.59287 

0.60004 

0.59561 

0.59970 

0.59820 

0.59918 

0.3 

0.53838 

0.65802 

0.61325 

0.65720 

0.64137 

0.65695 

0.64932 

0.65530 

0.65175 

0.65398 

0.65244 

0.65307 

0.4 

0.66166 

0.75106 

0.71912 

0.75083 

0.74020 

0.75010 

0.74616 

0.74970 

0.74795 

0.74912 

-  -  - 

0.74845 

0.74887 

Table  6 


Bounds  on  the  breakdown  point. 

Model  2.  Causal  filtering  operation  in  (59).  Independent 
size-m  batches  of  outliers. 

Upper  lines:  lower  bounds. 


ra 

1 

2 

3 

4 

5 

6 

0.002 

1.1434 

1.1339 

1.1334 

1.1334 

1.1334 

1.134 

0.01 

1.1454 

1.1360 

1.1355 

1.1355 

1.1355 

1.1355 

0.1 

1.1638 

1.1553 

1.1548 

1.1548 

1.1548 

1.1548 

0.15 

1.1729 

1.1648 

1.1644 

1.1644 

1.1644 

1.1644 

0.25 

1.1901 

1.1830 

1.1826 

1.1826 

1,1826 

1.1826 

0.3 

1.1986 

1.1918 

1.1915 

1.1915 

1.1915 

1.1915 

0.4 

1.2155 

1.2096 

1.2093 

1.2092 

1.2092 

1.2092 

Table  7 

Asymptotoc  mean  squared  error  at  the  nominal  model 
Model  1.  Causal  filtering  operation  in  (41). 


k 


m 

1 

2 

3 

4 

5 

6 

0.002 

0.14395 

0.07477 

0.05048 

0.03811 

0.03060 

0.02557 

0.01 

0.20683 

0,10940 

0.07433 

0.05698 

0.04528 

0.03788 

0.1 

0.40652 

0.22962 

0.15963 

0.12229 

0.09909 

0.08328 

0.15 

0.46731 

0.27014 

0.18937 

0.14568 

0.11835 

0.09964 

0.25 

0.56164 

0.33791 

0.24036 

0.18631 

0.15206 

0.12842 

0.3 

0.60122 

0.36851 

0.26394 

0.20534 

0.16795 

0.14206 

0.4 

0.67165 

0,42698 

0.31011 

0.24302 

0.19967 

0.16940 

Table  8 


Breakdown  point. 

Model  1.  Causal  filtering  operation  in  (41).  Independent  per 
datum  outliers. 


N.  m 

e 

1 

2 

3 

4 

5 

6 

0.002 

0.14395 

0.14395 

0.14395 

0.14395 

0.14395 

0.14395 

0.  01 

0.20683 

0.20683 

0.20683 

0.20683 

0.20683 

0.20683 

0.  1 

0.40652 

0.40652 

0.40652 

0.40652 

0.40652 

0.40652 

0.  15 

0.46731 

0.46731 

0.46731 

0.46731 

0.46731 

0.46731 

0.  25 

0.56164 

0.56164 

0.56164 

0.56164 

0.56164 

0.56164 

0.  3 

0.60122 

0.60122 

0.60122 

0.60122 

0.60122 

0.60122 

0.  4 

0.67165 

0.67165 

0.67165 

0.67165 

0.67165 

0.67165 

Table  9 

Breakdovm  point. 

Model  1.  Causal  filtering  operation  in  (41). 
Independent  size-m  batches  of  outliers. 
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Figure _ 2 

Bounds  on  tlie  Influence  I'unction 

Model  1.  Causal  filtering  operation  in  (39). 

£=0.01 

I^(v)  ;  Infl»ienee  t  unction  induced  by  the  optimal  at 
the  nominal  model  filter. 


FiRure  3 

Bounds  on  che  Influence*  Function 

Model  2.  Causal  filterinR  operation  in  (59). 

.:  =  0.01 

I°(v)  :  Intiuence  function  indiued  bv  the  optimal 
the  nominal  model  filter. 


0 


Figure  4 

Influence  Functions 

Model  1.  Causal  filtering  operation  in  (41). 

I°(v):  Influence  function  induced  by  the  optimal  at 
ni  ^ 

the  nominal  model  filter. 


Appendix  A 


Proof  of  Theorem  1 

tn  Ic 

The  class  F  and  the  functional  I(f  )  are  both  convex.  Thus,  for 
every  6  :  0^6 <^1,  we  have, 


^(1-6) f^ 


^  +  6  e  r 


I((l-6)f*^  +  ^2  ^  ^  ^  1(^2^  ~  I  ~  constant  w.r.t.6 


^Tn 

f  eF 


Thus, 


3^  I((l-6)f^  +  6  f2) 


r  rTvI: 

‘'y  —k  /IT 
7  k  (y  ) 


V2  [fi(y‘')f2(y''>i^ 


f2  (y'')J  [(i-6)fi(yS+<Sf^(y‘^)]^ 


from  which  we  conclude  the  statement  in  the  theorem,  due  to  the  continuity 
of  the  integrand. 

Proof  of  Lemma  1 

We  wish  to  find  some  density  function  fj^  in  F"*  that  attains  the  infimum 

k  in 

of  the  information  measure  I(f  )  in  (11),  where  the  nominal  density  f^  in  class 

F"*  is  the  convolution  of  the  Gaussian  densities  f  and  f  .  Applying  standard 

OS  ON 

variational  techniques,  we  conclude  that  if  f^  satisfies  the  above  infimum,  then 

k  k 

there  exists  some  supset  A  of  R  ,  such  that. 


^k.  k.  /I  %  /  k,  ^  k 

f*(y  )  =  (1-e)  f^  (y  )  ;  for  y  e  A 


(A.l) 


k  k  k  k  k 

For  y  e  [R  -  A  ],  the  density  f^  (y  )  satisfies  the  following 

differential  equation. 


k  ,  k. 


|V  I  ^  Af;  (y-')  ;  A  >  0 


(A. 2) 


The  general  solution  of  tin  li  i  f  f  en-nt  ia  1  equation  in  (A..’)  h.is  ttie  ton", 


WTOL’M  Jl  l-W  W  K^KW 


r^n^rr^TW’/  *«  >  u  w. » j  ^ '.'  ■»*■'  **■'  ^  ‘  T-i^.r»- v 


f*(y*^)  =  exp 


^  I  B^y^ I 


a.  2 


c  (y*^)  i  ;y*^e  [r*^-a*^) 


(A. 3) 


;  where  B  is  any  column  vector  in  R  that  is  not  orthogonal  to  the  vector  V. , 

k  k  ^ 

and  where  the  scalar  function  C(y  )  is  such  that,  V.C(y  )  =  0.  Imposing  now 

k  k  ^ 

continuity  of  the  density  function  f*(y  )  and  of  its  directional  derivative 

k  k  k 

^nf*(y  )»  everywhere  in  R  ,  we  conclude, 

Jt  * 


A*^: 


f^yS 


(A. 4) 


► . 


For  convenience,  we  now  select  B  =  (A.  3) ,  where  since  is 

positive  definite,  is  not  orthogonal  to  V^.  From  (A.  3),  (A.4)  ,  and  (A,l), 

k  k  I  T  k| 

and  requiring  that  f4.(y  )  is  continuous  on  the  boundary  |P.,  ()l)y  |  =  X,  we  fin- 

ally  obrain  the  density  in  (14).  The  expressions  (16)  and  (17)  evolve  from  the 

k  k  k 

requirement  that  /  dy  f  .(y  )  =  1.  The  estimate  in  (15)  is  easily  found  to  be 


/. 


the  optimal  at  mean  squared  estimate  of  the  information  datum  Xj^. 


b.l 


Appendix  B 


Proof  of  Theorem  2 


The  operation  in  (30)  has  the  general  form, 


^  a^  +g(y"'i  ?  x^,{xj}),  where  for  some  bounded  X,  g(x)  =| 


; |x|<X 


Xsgnx; |x| >X 


and  where  |Za.|  _<c.  Therefore,  |x  |  <  X[l+  |  J^a.]]  <  X(c+1);  Vn.  (B.l) 

i  1  n  i  1  — 


Let  E  {[X  -X  ]  }  denote  the  mean  squared  error  induced  by  the  estimate 
Uo  n  n 


X  ,  when  the  Gaussian  nominal  observation  process  is  acting.  Let  E  { [X  -x  j 
n  ®  U  n  n 


be  the  same  error,  when  some  process  in  class  F*"  in  (3)  is  acting  instead.  Let 


and  z”  denote  sequences  that  are  respectively  generated  by  the  processes  y 


and  y.  Given  some  set  A*^  in  r'',  and  in  conjuction  with  (B.l)  and  the  Schwartz 


inequality,  we  have. 


E  {[X  -X  ]^|z"ca"}=E  {X^|z"eA"}-2E  {X  x  | z^ea")  + 

U  nn  '  Uo”  ynn' 


+  E  { [x  j  z"ca"}  < 
U  n  — 


<  c  +  2E^^^{X^  I  z^eA^}E^^^f  [x  ]  ^  |  z'^ea'^}+E  {  [x  ]^|z'^ca'^} 
—  n'  n  '  y  n  ' 


i2  I  n^.n. 


<  c  +  2Xc^^^(c+l)+X^(c+l)^=[c^''^-(-X(c+l)  ]^=C 


(B.2) 


Due  to  (B.2),  and  considering  ergodic  and  stationary  observation  processes  in 


conjuction  with  F  ,  we  obtain:  Given  n>0,  there  exists  n  ,  such  that, 

o 


Vn>n  ;E  {[X  -x  ]^}<  (l-c+n)E{(X  -x  I^|(//i:Y  ( z!'|™,y  j^^)  >c  ]  <ne  ,y'^eR"}  +  cC 
oynn—  nn  m  i+l  i+1  ~ 


(B.3) 


;  where,  for  independent  m-sizo  outliers,  there  exists  some  (,^>0,  such  that. 


h.2 


<E  {[X  -X  ]  }  +€C;VC<e  ,Vn>n 
—  Mo  "  "  °  ° 


(B.4) 


From  (B.3)  and  (B.4),  we  conclude:  Given  n=  7r»  there  exist  n  and 

^  o 


e>0,  such  that, 


|E  {[X  -X  ]^}-E  {[X  -X  <  e  (2-  ^)C  +  ^  E  {[X  -x  ]^}< 

'  y  n  n  Po  n  n  '  -  2'  2  yo  n  n  - 


<  e|  C  =  6;  V  n>n  ,  V  £<e 
—  2  o  o 


2  6 

Thus,  given  6>0,  there  exist,  n  and  e:0  <e<min(e  ,  —  — )  ,  such  that, 

O  O  D  L* 


n  (y  ,y)  <e  implies  |e  {[X  -x  ]^}-E  {[X  -x  ]^}|<6  ;  Vn>n 


n,Pn  o 


y  n  n  y.  ‘  n  n 


The  proof  of  theorem  is  now  complete. 


Proof  of  Theorem  3 


Both  the  operations  in  (32)  and  (39)  have  the  form, 


kA 


_  ,  i+m-1, 

I  a.  g  (y.  ) 


;  where  |g(')|<A,  and  (Z  a.)  <  c.  Tlien,  subject  to  any  process,  h,  of  mutually 

i  1 

independent  m-size  batches  of  outliers,  each  occurring  with  probability  £ ,  we  have, 


^  nX,-X.  =  E  (Xf)  -2  I  a.  E  (X  p  (Y^^  + 

y  k  kA  Uq  K  i  1  y  k*^  i 

+  Z  I  a.  a.  E  {g(Y!'*^~S  g  (Y^'^'^t 
i  j  1  J  U  1  1 

=  E  {xh  -2(1-£)  Z  a.  E,  {x.  g(y!'^"^}-2e  Z  a.  E  f)C  g(Yf^”") } 

yo  k  i  1  Wo  K  1  i  1  h  k  1 


.  i+m-1. 


+  Z  Z  a.  a.  E  [g(Y^‘^  ^)R(Yj'^  S  I 
i  j  »  .1  bo  1  ) 


+  Z  Z  a .  .1 


+  4e  E  Z  a  a  E  {  |  g(Y  |  } 

1  j  1  J  P  1  J 

;  where , 

E^j{|g(Yi'^'^)g(Yj'^‘^)|}  <X^«» 

;  and  where  applying  the  Schwartz  inequality  we  have, 

E^^{lXj^llg(Yj'*™'^)l}  <  Ej^^{X^}Ej^^{g^Yj'^‘S}  1 
Eh{|\ll8(Yj'^"^)|}<Ej^^{X^}Ej^^{g^Yj'^"^}  <  <» 


Applying  inequalities  (B.6)  and  (B.7)  to  (B.5),  we  obtain, 

=  4c  (E  a.)X  +  E  a. ]  =  cC 

i  1  i  1 

;  where  C  =  4A(E  a.)  +  E  a  ]  «». 

i  1  i  1 

Now,  given  6>0 ,  we  select  G  =6  C  \  to  obtain. 


^  (u  ,M)  <e  =6  c  ^  ^|e  Hx  _0  ]^}  _E  {[)(  1^)| 

n,p  o  U  k  kA  Uo  k  kA 


and  the  proof  of  the  theorem  is  complete. 


(B.5) 


(B.6) 


<co 

(B.7) 


(B.8) 


;  Vk 


b.4 


Proof  of  Theorem  4 

We  will  prove  the  theorem  in  steps.  Let  r^  denote  the  pth  autocorelation 

coefficient  of  either  the  sequence  {Z.  =  or  the  sequence  {e  =  P^ 

1  m  i-nri-1  ^  in  i-n  ’ 

at  the  nominal  Gaussian  model.  That  is,  either  r  =  E{Z.  Z  I f  } ,  or  r  =  e{e  E  If  } 

p  1  i+p '  o  p  1  i+p '  o 

Let  then  c  be  the  constant  in  (26) ,  and  let  us  then  denote, 


2 

o 


al  =  [iVl 

P  p 


(B.9) 


Let  L^  denote  the  pth  autocorelation  coefficient  of  either  the  sequence 

{g„  (Z.)},  or  the  sequence  {g„  (E.)},  for  g_  (.)  and  g  (.)  respectively  as  in 

r  ,m  1  b , n  i  r  ,m  b,n 

(47)  and  (48),  and  at  the  nominal  Gaussian  model.  That  is,  either 


L  =  E{g  (Z.)g_  (Z..  )lf  },  or  L  =  E{g^  (E.)g^  (E._^  )|f  ). 

p  °F,m  1  °F,m  i+p  '  o  p  "S,n  i  ®S,n  i+p  '  o 


Let  g(.)  denote  either  g„  (.)  or  g„  (.),  and  let  f(x)  denote  the  zero  mean, 

r ,m  b ,n 

2 

variance  o  Gaussian  density  function  at  the  point  x.  Let  X  denote  the  truncation 
threshold  of  the  function  g(.).  Then,  for  (j)(x)  and  iKx)  being  respectively  the  density 
and  the  distribution  functions  at  the  point  x,  of  the  zero  mean  unit  variance  Gaussian 
random  variable,  and  for  c  as  in  (26),  we  directly  obtain. 


{2[l-c 


(c)  -  2c(|)(c)+  2c  -l) 


(B.IO) 


-Tp  [2  <I>(c)-l]  ;  IpI  >  1  (B.ll) 

(k) 

Denoting  now  by  (J)  (x)  ,  the  kth  derivative  of  (t)(x),  at  the  point  x,  and 

applying  the  Taylor  theorem,  we  have  that. 

Given  n,  there  exists  t  ;  0  •  t  <1,  such  that, 

n  n 


2(n-l) 


k=0  '  P ' 


k+2  k+2 

Y_  X 


o  (k+2).' 
P 


2n+l  2n+l 

^4,(2n-l)A  - _ 

1  0  no  )  2n+l  , V , 

'  p  p  /  Op  ( 2n+l ) . 


(B.12) 

Substituting  (B.12)  in  (B.ll)  ,  and  via  some  straightforward  transformations. 


we  obtain. 


p  >1;  L  =20 
-  P  P 


t  ^  n-j.  2k+l  2k+l 

I  P  ^  p/  k=l  \  P  /  o^‘^"^N2k+l) 

./2n-l)  A..  ,,  IpM  V  - 

\  n  o  J  2n+l ,  ~  .  1  \  I  1 

'  P  p  /  o  (2n+l) !  I 


(2n-l) 


\%  /  0^'"'^^  (2k+l)'. 
2n+l  2n+l 


-r  [2<P(c)  -1]  = 
P 


^  “2n^  '  /  ’‘^"■'^g(x)f(x)  /  A  +  t 

Op  (2n+l)  I  J  -oo  * 


V  \ 

no/ 
P  / 


(B.13) 


;wherf‘,  for  c  as  in  (26), 


b  (n)  =  r  [24>  „  ,-7. 


-1)  12<Kc)  -1] 


n-1  r  T 

s  [■;  ''1 


■k  ^  (2k-l) 


c  _  \  )  2<I)(c)-l 

I  ?  k! 


^  2H-1  „k-^+l  I  k.' 


(2k+i):  (i’-i): 


_,-2(k-(’+i)  ]/ 

k!  (2(’-i) :  (  'b.u) 


b.6 


2n+L  ,  >,c/  \  A^^n-l) 
X  g(x)f(x)  4)' 


<  I  max  1  4) 


la  no/ 

\  p  p  / 


dx  < 


(2n-l) 


/  A_  ^  ]  [l2M) 


f(x)dx  = 


;  where. 


=  C 

"  2"  nl 


.(2n-l),  ,  ,  2"  nl 

C  =  max  4>  (x)  <  - 

"  X  2n-l 

*  V  2tt  e 


(B.15) 


(B.16) 


From  (B.13),  (B.14),  (B.15),  (B.16),  and  (B.9),  we  conclude. 


L  -  b  (n) 
P  P 


1/2  Y 


- =  B  (n)  ;  IpI  >  I  (B.17) 

e(l-Y  )  P 


2  e 

;  where  b  (n)  is  as  in  (B.14).  We  note  that,  if  Y  »  then  given  p,  given 

p  p  i+e 

2  e 

C  >  0,  there  exists  n  ,  such  that,  B  (n)  <  r  V  n  >  n  .  Thus,  if  Y  ^  r,'  ; 

op  p  --  p  op  p  1+e 

V  IpI  >  1,  and  if  is  such  that,  then  given  p,  given  ?;  >  0, 

there  exists  n  ,  such  that  B  (n)  <  C  r  ;  V  n  >  n  ,  and  {n  }  is  a  decreasing 
op  p  —  p  op  op 

sequence  with  increasing  jpj.  We  thus  conclude  that,  if  >  ’'|p|+l^^^ 

2 

<  Y^g  !  ^  IpI  ^  then,  given  C,  >  0,  there  exist  a  decreasing  with  increasing 
IpI  sequence  and  a  positive  integer  N,  such  that. 


b  (n)  -  i;r  <  L  <  b  (n)  +  ^r  ;  Vn  >  n  ,  for  given  p 
p  P“P“'P  P  op 


b  (n)  =  b  =  r  12<l>(c)-l]  ;  V  p  >  N 

P  P  P 


(B. 18) 


Let  us  now  denote  by  C((jJ)  ,  either  D(a))  in  (45)  or  A(tj)  in  (46).  Let  us 


denote  by  H((jj)  ,  either  H_  (to)  in  (45)  or  (to)  in  (46).  Let  also  B((o)  be 

F,m  S,  n 

the  Fourier  transform  of  the  sequence  {b  (n  );1p1  N-L,b  ;!pl  >  N,L„}  in 

p  op  ~  p  \J 

(B.18)  and  (B.IO).  Let  e(f  ,x_)  denote  either  the  error  e(f  ,x__)  in  (45),  or  the 

o  U  o  Ur 

error  e(f  ,x„  )  in  (A6)  .  Then,  due  to  (B.18)  and  (B.IO),  we  directly  obtain, 
o  OS 


e(f  ,x-)  -  e'^(f  ,x_)  <  C(2Tt)  ^  (2Tr)  /  |  |  C((o)  |  |  ^f  (to) dto 

O  (J  o  U  —  /  s 


I  I  c((0)  I  I  dto  1  f^((o)dto 
-Tt  J-V 


(B.19) 


;  where. 


e  (f^,XQ) 


(2tt)  V  /  f^(to)dto  -  2(24>(c)-ll  I  Re(C(to)  )H(to) )  fg(to)dto 


+  /  I  lC(to)  1  I  ^B(to)d(o|. 


(B.20)  (B.20) 


B(to)  =  0^  {2[l-c^l  t  (c)  -  2c({)(c)+2r^-l }  +  ^  b^ ( n^p) + 

1  '■  j  p  I  <  N- 1 


+  124  ((  )-1]‘'  y;  r 


> 


[2<J>(c)-l]  (ui)  +  a^)24'(c)  [3-c^-2<J)(c)  ]  -2c<p(c) 
s 


+  2c^-2{  +  (n  )  -  [<t>(c)-l]^  r  [  (B.21) 

^  '  p  op  p  ' 

1<1p1<N-1 

The  statement  of  the  theorem  follows  directly  from  (B.19),  (B.20) ,  and 
(B.21). 


Proof  of  Theorem  5 


(i)  The  eigenvalues 


polynomial  equation, 
k  k-1 

X  -  X  a^  -  . 


of  the  matrix  A  in  (58)  are  the  roots  of  the 


.  .  -  X  a 


k-1 


=  0 


Due  to  the  assumed  stationarity,  their  magnitudes  are  all  less  than  one.  1 

2in 

the  eigenvalues  of  the  matrix  A  have  also  magnitudes  less  than  one. 


Let  us  now  consider  the  matrix  C  in  (61)  as  a  function  of  the  design 

parameter  m,  and  denote  it  then  C  .  For  m  varying,  we  easily  find  that, 

m 

C  ^  =  C  +  (I  -  C  )  a"*  h  a”™  (B.22) 

m+1  m  m  —  -m 


;  where. 


h 


I  B 
-m 

2 

O  (-m)  +  r 


b.9 


and  where  the  expectations  in  (B.23)  are  taken  at  the  nominal  model.  Also, 


Eq  b  b- 


^  (0)  +  r^ 


(B.24) 


It  is  easily  seen  that  the  matrix  in  (B.24)  has  one  eigenvalue  between 

zero  and  one,  and  that  the  remaining  eigenvalues  equal  zero.  Due  to  (B.22),  and  by 

induction,  the  same  assertion  holds  for  C  ,  m  >  1.  The  magnitudes  of  the  eigenvalues 

m 

of  the  matrix  Q  are  then  also  less  than  one,  and  the  same  assertion  holds  for  the 
matrix  P.  Hence,  the  equations  in  (62)  have  solutions. 


(ii)  Let  us  consider  the  vector  Z  in  (61),  and  let  us  define. 


X 


u 

—  -m 


Y 


.m 


-m 


(B.25) 


From  (59)  we  then  easily  obtain, 

X  =  Y  +  a"'  -  g  ^  (Z  +  C  a"*  fl)  (B.26) 

—  —  ~  ^  F,m  —  — 

Under  the  nominal  model,  the  vectors  Y  and  ^  are  jointly  Gaussian,  and 
independent  of  the  vector  Let  ((  (x)  and  ,'^q(w)  denote  the  density  functions  of 

A  it 

the  vectors  X  and  U  at  the  nominal  model,  and  at  respectively  the  vector  points  x 
and  w.  Considering  expectations  at  the  nominal  model,  let  us  define. 


■  Mt''  •  ^ X  I  } 


A  (x,  w)  = 


/, 


1  T  -1 

dz  exp(-  —  z  S  z) 

2  ~  z  — 


iS 


1/2  1/2  Y  fH  +  £  p  ■*■  CA  w)  -A  oj  -Rz 


Jx+gr  (z  +  CA^O))  -A^cj  -  Rz])  (B.2 

y/Z  —  m  —  —  —  — 


"F,m' 


;  where  and  are  given  by  (61).  In  the  sequel,  we  will  use  the 


following  theorem,  where  for  x  =  (x^^ ,X2.  •  •  •  ,Xj^]  ,  we  define. 


|x| I  =  max  |x^| 
i 


(B.28) 


Theorem  B 


k  £  k 

Let  f(x,v)  :  R  x  R  -*■  R  be  a  measurable  function,  satisfying  the  following 


condition,  for  some  positive  real  scalar  function  h(v). 

'^rX 


|f(x,v)  -f(x^.v)||  £  II  Jc-x^'ll  h(v)  ;  V  x,  x^eR‘^,VvER'^  (B.29) 


Let  n  ^  0}  be  a  stochastic  process  in  R  ,  defined  by  the  recursive 


equation. 


4+1  ■  f<4-  V  •  "  i  “ 


(B.30) 


;  where  n  ^  0}  is  an  i.i.d.  process  in  R  ,  and  where  is  independent 


of  X^,  for  all  n  ^  0.  Let  V  be  absolutely  continuous,  and  let  then  p(v)  be  its 


density  function  at  v.  Let  p(v)  be  such  that, 

h(v)p(v)dv  =  ^  <  1 


/, 


(B.31) 


t 


Then,  the  process  {X  ,  n  >  0}  is  asymptotically  stationary.  Furthermore, 


the  asymptotic  distribution  of  X  ,  for  n  ^  is  the  same  for  any  initial  distribu- 

— T\ 


tion  of  X^,  and  depends  only  on  the  function  f(*,*)  and  the  density  p(y) . 


b.ll 


Proof 

From  (B.30)  we  conclude  that  {X^^)  is  a  Markov  process.  Thus,  to  prove 

asymptotic  stationarity ,  it  suffices  to  show  that,  given  any  distribution  for 

k 

Xp,,  the  distribution  of  X  converges  weakly  to  a  unique  distribution  in  R  , 

— u  — n 

as  n 

Let  Uq(x)  be  an  arbitrary  density  function,  VxeR  .  Let  then  the  sequence 

{m  (x) ,  n  >  0}  be  defined  as  follows. 

n  —  — 


^n+1  = 


)  =  r  A  (x,w)  Mj, 

J  k 


(oj)  doj 


(B.32) 


;  where  A(x,w)  denotes  the  conditional  density  function  of  x,  given  w, 
when  X  =  f(w,_v),  and  where  is  independent  of  V,  and  p(v)  is  the  density 
function  of  V  at  veR^.  Let  us  now  define  the  sequence  i 


as  follows. 


A^^^  (x.io)  =  A(x,a)) 


(B.33) 


,(n+l) 


(x,w)  =  /  A*'  ^(x,z)  A'^  '{z,(ii)dz 


Then,  we  can  write. 


•  j 


A^”\x,u)  Uq(w)  dw 


(B.34) 


To  show  weak  convergence  of  the  sequence  {u^(x)},  we  need  to  show  that 
there  exists  a  density  function  u(x)  ;  xeR  ,  such  that,  for  any  continuous  and 


bounded  function,  g(x)  in  R  ,  we  have. 


/  “n 

k 


(x)dx 


j  sM  u 


(x)dx 


(B.35) 


Let  us  define  the  sequence  {g  (x) ,  n  >  0),  as  follows, 

n  —  ” 


g^(x)  =  g(x) 


i,(x)=  r 

J  Ir 


A(z,x)  g  ,(z)dz 
- n-l  —  — 


(B.36) 


Then, 


/  gW  Mj^(x)dx  =  /  Sj^(x)  yQ(x)d 

Ak  Jk 


X  (B.37) 


Let  us  define, 


u  =  sup  (5  ^  sup  lg„U)  -g„(w)|) 


n  —  n  — 


(B.38) 


loj-z  I  |<  6 


Without  lack  in  generality,  we  will  assume  that  the  quantities  {u  } 

n 

are  all  finite  (this  is  true  if,  for  example,  the  functions  g  (x)  satisfy  a 

n  — 

Lipshitz  condition).  From  (B.30)  and  (B.36),  we  obtain, 


=  y  P(v) 


(B.39) 


From  (B.38)  and  (B.39),  we  conclude. 


u^  <  r  i  sup  (6  ^  sup  lgjj_j(f(x.z))  (w,v))  1  )|p(v)dv 

J.  6>0  ||x-u||<6 


h(v)p(v)  |sup  ([6h(v)]  sup  I  g^.j  (ii^~gn_l  )|dv 

6>0  I  I x-<i)|  j< 6h(v) 


“n-l  / 

Ak 


)dv  =  Cu  1  u  , 

—  n-l  n-l 


(B.40) 


From  (B.^O),  we  conclude  that  u  0,  as  n  and  that  g  (x)  -♦  g(x) 

n  n  —  - 


constant  on  R 


as  n  Thus, 


g(x)  u  (x)dx  = 
n 


)dx - -  constant 


(  B .  I) 


b.l3 


Due  to  (B.29),  the  sequence  {u^(x)}  is  tight.  Thus,  there  exists  a 

subsequence  (m  (x)},  and  a  density  function  u(x)  in  R  ,  such  that,  for  every 
"i 

continuous  and  bounded  function  g(3c)  ,  we  have. 


(x)g(x)dx 


n.-^ 

1 


U(x)g(x)dx 


(B.42) 


From  (B.41)  and  (B.42),  it  immediately  follows  that. 


g(x)Uj^(x)dx  - - 


I  g (x ) U ( x) dx 


and  the  proof  of  the  theorem  is  now  complete. 


Let  us  apply  now  theorem  B,  for  expression  (B.26).  In  this  case, 

V  =  [Y,Z]^  ,  £  =  2k 

=  Y  +  A™  -  g„  (Z  +  C  a"*  «) 

—  —  —  —  r  ,m 

;  where,  for  denoting  the  absolutely  largest  eigenvalue  of  B,  we  have, 

II  f(x,v)  -f(n,v)  II  <  11  A*"  (i-c)(x  -  n)||  < 


The  function  M(w)  in  (B.46)  is  analytic,  and  possesses  Taylor  exnantion. 


In  particular,  applying  the  Taylor  theorem  we  write. 


M^j  (w)  =  j  W  +  [V^M^j(w)  |u  =  +  2  ^  £]w  ;  for  some  Jj: 

0  £  jJ  £  w  (B.A7) 

;  where  we  can  easily  find  that, 

M(0)  =  P 

(B.48) 

(A^)™A™  w  a"*  w  <  [V^  (w)  I  w  =  U.]  w  (A^)™  A™  u  ;  V  a 

The  expressions  in  (B.48),  in  conjuction  with  the  equality  =  0,  and 

expressions  (B.45)  and  (B.47),  give  the  inequality  in  (63). 


;  where  the  inequality  in  (B.49)  evolves  from  the  fact  that  the  magnitudes  of  the 
eigenvalues  of  the  matrix  P  are  all  less  than  one. 


Appendix  C 
Proof  of  Theorem  6 

p  j> 

Let  e„  r  ,  „(f  ,6,x-)  and  e  ,  ,  denote  respectively  the 

mean  squared  errors  e(f^,6,XQp)  and  e(f^,6,XQg),  when  the  correlation  coefficients 

r  (k)  and  r  (k)  are  substituted  by  the  bounds  (52).  Then, 

®F,m  ®S,n 

we  easily  find, 


4.(n 
F  pF 

t  \  c<f  •‘5,x_)  =  E{xJ|f  }  +F  (6) 
Ns,{nps}.S  o  0  O'  o  S 


(C.l) 


A  pA  A  pA 


;  where  er  r  \  Af  ,x^);  A=F,S  are  the  error  bounds  in  theorem  4. 

Njln  ./»A  o  U 
A  pA 

It  can  be  easily  seen  from  (70),  (71),  and  (73),  that  e(fo.6.Sop). 

e(f  ,6,x_„),  F„(6),  and  F  (6)  are  all  monotonically  increasing  with  increasing  6, 
o  OS  F  S 

and  are  all  continuous  and  differentiable  as  functions  of  6.  Also, 


e(f^,0,S„p)  <  E{X^|t^).  e<f„.0.S„3)  '  E{X,|f^),  and. 


(C.2) 


e(f  =  E{Xnlf„}  +  uV(0) 


o’  ’OS' 


o  n 


*  *  -1 
Thus,  the  breakdown  points  5„  and  6^  exist,  are  unique,  and  are  strictly 

^  F,m  S,n 

between  zero  and  one.  In  addition,  if  can  be  easily  found  that  the  functions 


G,,(6)  ^  e(f^,6,XQp)  -EiX^lfJ  -F,(6),  and  C^((i)  e(f^,<S.x^^)  -  K(xjf,^l  -F^(^) 


O'  o  F 


o’  ’OS' 


O'  o  S 


are  such  that, 


Gp(l)  =  Gg(l)  =  0 


|Gp(6)|  <  |Gp(0)|  -  |e(f^,XQj,)  ,{n 

F  pr 


CC.3) 


103(6)!  <  103(0)!  -  le(f„.«„3) 

S  pb 


Due  to  (C.3),  and  via  theorem  4,  given  >  0,  we  can  find  integers  and 

Ng,  and  sets  ^rippil  1  |p|  1  <  |p|  £  that, 

|g  (6)  I  <  Ci;V6,  |g  (5)  I  <  c;  ;V6.  Then,  for  C,  small  enough,  we  can 
r  1  S  1  1 

2  2 

also  have,  F  (0)  <  0  and  F  (0)  <  0,  while  at  the  same  time  we  have,  F  (1)=X  D  (0)>  0 
F  0  r  m 

2  2 

and  F-(1)=d  A  (0)  >  0.  Thus,  the  functions  F_(6)  and  F  (6)  will  then  have  unique 
bn  r  b 

zeros,  6°„  and  6°  .  Due  to  the  continuity  of  all  the  functions,  e(f  ,6,x^_) 

F,m  S,n  o  OF 

e(f  ,6,x^  ),  F  (5),  and  F  (6),  with  respect  to  6,  given  C  >  0,  we  can  find 

O  Ub  V  b 

>  0,  such  that  |Gp(6)  ]  <  C^;V6  and  10^(6)!  <  C^;V6,  gives: 


1 6*  -6°  1  <'  C  and  l6*  -6°  |  <  C- 

F,m  F,m  S,n  S,n' 


Proof  of  Lemma  2 

Let  us  define, Y  ,  X,  and  as  in  (B.25),  and  let  us  also  define. 


A  V*  ,  uT.  A^i+i  ,  , 

1=2^  b  B  [  U  -A  n  } 
^  •  _  _  1  1 


i=-m+l 


(C.4) 


z  •  53  b,  “■ 


Then,  for  C  as  in  (61),  the  optimal  at  the  nominal  model  filtering  operation 


can  be  written  as  follt'ws. 


;  where  is  independent  of  Y,  Z^,  and  Z^,  where  Y  and  Z^  are  independent  of  Z^, 
and  where  the  vectors  Y  and  Z^  are  zero  mean  and  jointly  Gaussian.  In  addition, 
denoting  by  G(M,E)  the  mean  M  and  covariance  matrix  Z  Gaussian  distribution,  the 
vector  ^2  the  following  distribution,  in  the  persence  of  the  m-size  block 
outlier  model. 

(1-6)  G  (O.r^N)  +  6  G  (vp.O)  ;  0  <  6  <  1  (C.6) 

Due  to  (C.6)  and  the  fact  that  the  distributions  of  X  and  are  asymptotically 
identical,  we  easily  find  from  (C.5), 


E{x}  =  -5v  [I  -  (I  -  C)  A*"]  \i 


-1 


(C.7) 


From  (C.5)  and  (C.7),  we  also  find» 


m 


e{x  X^}  =  (I-C)  a“  E{X  X^}  (A^)  (I-C)^  + 


+  (1-6)  (E{(Y-Z^)  (Y-Zj)^}  +  r^  N)  + 

+  6(E{(Y-Zj^)  (Y-Zp'^}  +  v^  u  M^)  + 

+  6^v''((I-C)  A™  fl-(I-C)  A*"]  U  /  + 

+  y  [I-  (a'^)'"  (I-C)"^]  (a'^)”  (I-C)^) 


(C.8) 


Solving  (C.8),  we  finally  find,  for  E^{X  X^}  =  E{X  X^}  at  6=0, 

E{X  X^}  -  E  {X  X^} 

1°  (v)  =  lim  - ^ -  = 

6->0  6 


T  7  TUI  jl 

[(I-C)  a"*]  [v^  y  m  -  r^N)  [(A^)  (I-C)  ]  (C.9) 

i=0 


U 

;  where  the  infinite  sum  in  (C.9)  converges,  siiire  the  eigenvalues  of  (I-C)A 


have  magnitudes  that  are  strictly  less  than  one. 


Proof  of  Theorem  7 


(i)  Let  2^,  end  ^2  be  defined  es  in  eppendix  end  considering  expectetions 

et  the  nominel  model,  let  us  define. 


>y/z  =  e{[y  -  e{y|z^}]  [Y  -  E{y|Zj}]'^} 


R  =  S  ^ 

1  UZ 

A  (x,a))  =  (2Tr)“*^  |s  , 

V -  y/z^ 


-1/2 


/■"  / 


dz  exD(-2  ^z^S  ^z) . 

k  ^1 


(C.9) 


,-l. 


,T  „-l 


.exp(-2  [x  +  £  (^-f-vry^+CA  (jj)-A  w-R^z]  S  ,  [x  +g  (z+vy+CA  w)-A  w-R, z]) 

i  y/z^  r ,  m  —  —  —  —  1 — 

Given  6,  end  the  m-size  block  outlier  model  et  level  v,  the  vectors  Y,  Z,  end 
Z  ere  still  independent  of  the  vector  Then,  for  |{„(3c)  end  respectively 

denoting  the  density  functions  of  X  end  Q,  for  A(x,w)  defined  es  in  (B.27),  eppendix  B, 
end  for, 

=  (l-6)A(x,u)  +6A^(x,w)  (C.IO) 

we  can  write, 

(C.ll) 


~  If  ~ 


It  cen  be  eesily  found  that  theorem  B,  in  the  proof  of  theorem  5,  appendix  B, 

applies  on  (C.ll)  with  h(v)  =  |y  (A™(I-C])|  <  1.  Thus,  the  densities  ((„(x)  and 

n\3X  X  — 

(C.ll)  are  asymptotically  identical.  From  (C.ll),  we  then  obtain, 

E{X  x'^}  =  /  [(1-6)  M(w)  +  5M^(w)]  i(^(w)  dw  (C.12) 


;  where  M(w)  is  as  in  (B.46),  appendix  B,  and  is  bounded  as  in  (B.48),  and  where. 


M^(u)  = 


L 


X  X  A^(2<,w)  dx 


(C.13) 


Applying  the  Taylor  theorem  on  the  analytic  matrix  function  M  (co) ,  and  similarly 

V  — 

to  (B.A7),  for  the  matrix  M(w) ,  we  find  the  following  expression,  where  A  <  B  means 
that  each  diagonal  element  of  the  matrix  A  is  bounded  from  above  bv  the  corresponding 
diagonal  element  of  the  matrix  H. 


(C.14) 


+  A*"  w  (A^)"  +  [VM^(a))  u  >  M^(u)  1  +  A®  u  (A^)™  -*• 


+  [VM^(w)|^  ^  q]u  -  R^(w) 


;  where  R  (w)  =  {u^(A^)'’'q .  ,  a”'w  ;  i,j  =  l,...,k}.  Let  us  now  define  |J°(x) 
V  ij ,v  X 


and  as  follows. 


($v(x)  = 


=  /  ^(x 

J  k 


,u)  du 


(C.15) 


;  where  from  the  proof  of  theorem  5,  we  have  that  and  are  asympto¬ 

tically  identical.  Then,  considering  also  the  densities  in  (C.ll),  we  easily  find. 


I  (v)  =  /  M((x))  [  £im  5"\,f  (£j)  -  ,{°(w))]  dw 


Iw  +  f 


[M^(u)  -  M(w)  du 


(C.16) 


;  where  from  theorem  5  we  have. 


uj  i5p(w)  du  =  0 


3^  < 
OF  - 


M(w)  dw  <  S“p 


and  where, 


w  w^[.£im6  ($_ 

ik'”  - 


((D)  -  |J„((D))]  dCD  =  I  (v) 
~  W  —  —  ra 


From  the  above,  and  using  (C.14)  on  (C.16),  we  finally  find  the  result  in  (80) 


(ii)  The  results  follow  easily  from  the  bounds  in  (B.47),  appendix  B,  and  the  bounds 
in  (C.14),  by  setting  v  -»•  <»,  and  by  substituting  1-6  by  (1-6)"',  and  6  by  1-  (1-6)"'. 


In  particular,  we  then  find, 

f„(6)  =  tr(S^(6)  -  S  )  <  tr(E{X  x”^}  -  E{U^  ^))  <  tr(s''(6)  -  s  )  =  f,  (6)  (C.17) 

2  m  O  —  -  —0  ^  —  m  O  1 

;  where  the  functions  fj^(6)  and  f2(<5)  are  trivially  found  to  have  unique,  strictly 

u  £. 

positive,  and  strictly  less  than  one  roots,  6  and  6  .  The  inequalities  in  (82) 

m  m 

follow  then  trivially  from  (C.17). 
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